Jack - the SAN is independantly connected to the network. The HP3000's
VA is not directly connected to the network but is part of the HP's
internal storage. Hidden in my original question was the implication
that the SAN was broadcasting some special type of network traffic which
was somehow flooding the network and the HP3000 or its own VA was having
to spend time dealing with interrupts from these messages.
Here's an update on my original post.
Thanks to all who responded with suggestions - especially Craig Lalley
who makes me realise as soon as you ask one question, then you realise
there's a whole raft of other (more technical) questions that need to be
answered first before the context of the problem can be understood and
resolved.
For now the problem has gone away - but we're still not absolutely sure
what the original cause was (we're waiting for some logfiles to be
analysed by HP).
Here's a status report (an edited version of another email).
<clip>
Just for reference, we're thinking that the problem all along was the
failure of one of the Mirrored disks on the VA and - for whatever reason
- it took a very long time for the new drive to resync itself after a
new disk was swapped in by an HP engineer.
Just a quick rewind of the problem....
a) we started noticing the end of day was taking a bit longer than
normal
b) I spotted that the disk to disk backups were partly to blame (had
suddently started taking twice as long for the same amount of data)
whilst the backup to tape (LTO) of the same data was not affected.
c) Other overnight programs that write to disk heavily were also
affected whilist jobs that simply read data were performing normally.
We then tried to work out what the problem was and it just so happened
that a new SAN was turned on (and connected) to the network the same
weekend that the HP started to degrade.
Since then, we have been trying to isolate the cause and our operator
spotted a few days ago that one of the lights on the VA was amber and we
got HP to swap the disk unit over. In the meantime, we also asked the
network team to turn off the SAN and turn it back on again comparing the
results of the main disk to disk backup.
As I say, the jury's dismissed the SAN as the root cause of the problem.
<end clip>
-----Original Message-----
From: Jack Connor [mailto:[log in to unmask]]
Sent: 01 June 2007 13:21
To: Tony Summers
Subject: RE: [HP3000-L] New SAN affecting our production HP3000
Tony,
Would the SAN traffic be going through the same fiber switch as the VA
traffic to the 3K?
It's possible that there's a bandwidth problem on the switch...
Jack Connor
-----Original Message-----
From: HP-3000 Systems Discussion [mailto:[log in to unmask]] On
Behalf Of Tony Summers
Sent: Friday, June 01, 2007 7:17 AM
To: [log in to unmask]
Subject: [HP3000-L] New SAN affecting our production HP3000
Although the jury is still out as to the real reason, we've been having
performance problems on our production HP3000 immediately after a new
SAN was connected to the network. (The SAN is currently being
commissioned as part of our general server consolidation, but is not
used by the HP which has its own private VA).
We're planning to run some controlled experiments over the next few days
(turning the SAN on and off and comparing results) but has anyone else
experienced something simliar where an HP3000 seems to suffer as a
result of unrelated kit being added to the network ?
For reference, HP are looking at the VA's logfiles, and we also
suffered a disk problem on the VA which was swapped out in the same
period. For that reason, we still don't know for sure whether the SAN
can be blamed or whether disc mirroring on the HP3000's VA was simply
taking some time to catch up.
However, when we turned the SAN off last night, then the HP's
performance returned to normal. I doubt it's co-incidence - but I
would like to be proved wrong.
Regards
Tony Summers.
The contents of this email are confidential to the intended recipient
and may not be disclosed. Although it is believed that this email and
any attachments are virus free, it is the responsibility of the
recipient to confirm this.
Details of Smith & Williamson group companies and Nexia Smith &
Williamson Audit Limited and their regulators (where applicable), can be
found at this URL
http://www.smith.williamson.co.uk/disclosure
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
|
