This is the start of a lot of writing, because if I put it down in words, it’ll make me believe that I’m in control and I know what I’m doing. I promise you, I’m not and I don’t. Most of this will be disjointed sentences adapted from my hasty notes and vague memories, but some of it might prove useful, or at least entertaining.
It’s also a manual to myself in case I ever go through this again in the future, because there’s a lot to forget and a lot I’ve learnt the hard way.
So. I’m making a new Hyper-V virtual server infrastructure.
My network is 400ish machines and about a dozen servers. The main DC is a Pentium 4 running 2k3. Suffice to say, it’s Not Really Good Enough. I want to move to Windows 7, so I have to move to 2k8R2, so I may as well do it properly. I’m not redoing AD altogether, the domain will just migrate as it’s in decent nick, but the hardware infrastructure should eventually be replaced altogether.
To that end, I’ve got the following kit to play with:
- 1 x HP DL160 G6 – to be a physical DC and (nominally) host SCVMM & DPM
- 2 x HP DL380 G7 – to be virtual hosts in HA failover pair
- 1 x QSAN Fibre Channel SAN (dual controllers, 6 x 300GB 15k drives)
- 2 x Fibre Switches
- 1 x LTO5 tape autoloader
- 2 x UPS (1x 3000VA, 1x1500VA) to power everything + the rackmount VLE we have already
This first post is about putting it all together.
Putting It All Together
Firstly: racks are bloody heavy and awkward. Secondly: they’re even heavier and more awkward once everything is in and you’ve taken the wheels off, so make sure you like where the rack is.
Sit and draw where everything will go, leaving room for expansion and room for airflow. Try and think about the length of cables as well, otherwise you might not be able to plug your autoloader into your server. Be warned though: no matter how much you plan and measure, the last thing to go in won’t quite fit, and you’ll have to empty everything out again to adjust a vertical strut by half an inch to put everything back in correctly. After fitting everything else in my rack, the DL160 G6 at the top stuck out by half an inch. I could’ve left it as it was, but the point of all this is to do it right, so everything came out, struts got moved, and eventually it all fitted.
You should sit down and draw out how everything will connect as well, because doing this will remind you to buy the fibre patch cables and SFPs you need and forgot to budget for. You will also remember that fibre switches do not run off hope and determination, and require a power cable, so you should get another UPS to give you the required number of sockets. All these surprises do not necessarily come cheap. An actual drawing is probably unnecessary, but it’s worth stepping back and considering it all properly at any rate.
When you come to plugging it all in, run power cables up the right hand side and data cables up the left, or at any rate, run them separately: it looks neat and it prevents interference.
Keep all the tie tags from everything you open to hold cables in place while you work. You can use proper cable ties later when everything is definitely in place; to be honest, at this point I’m not sure if I will, it’s neat already. It’s worth keeping cables grouped by where they go, though, rather than just tying everything together at set points.
Label every cable at both ends with the server and port number, no-one ever complained of too much knowledge and you’ll be glad of it later.
Make sure you don’t bend fibre too much, it likes big, slow curves for the light to bounce around. Keep the rubbery bits from SFPs and the little white caps from the fibre patch cables in one of the many spare bags you should have lying around by now, they’re useful when you need to reconfigure your cabling.
Ideally, turn your network off for a day and recable your main cab so that it’s nice and neat and ready for all the new cabling. You are not likely to have this luxury. Weep instead as you try and squeeze cables into spaces too small for your fingers, and resign yourself to the fact you will have to spend 2 days over the summer redoing it all – which probably isn’t a bad idea anyway, as the old servers will have been decommissioned by then.
Once everything is plugged together, the doors are back on your rack, everything is neat and ready, check the UPS power levels for your old kit, and make sure you know where the fuse board is. That way, when you turn the new UPS on and it trips the ring out, you can get it sorted before all your other servers fall over.
At this point, it’s time to put your screwdrivers down, leave the RJ-45 crimper on the side and go for a cup of tea and a biscuit before you start on installing Windows. You’ve earnt it, soldier.