| Author |
 Topic  |
|
|
jaxdave
Honorable But Hopeless Addict
USA
2212 Posts
Status: offline |
 Posted - 07/25/2006 : 10:14:34 PM
|
How can we have this talk ?
I am currently drawing up a Business continuity plan for my enviorement. In this enviorement I have some Vmware ESX servers , Fibre channel SAN,Exchange,SQL,ORacle , Linux Oracle Forms servers, Citrix, AD , so and so on.
Alot of the fail over redundency can be handled with products such as visioncore replicator,legato etc. My quesiton is how do I have a DC in waiting out at my cold or hot DR site that holds my AD schema ,users etc without having it be a live DC ? I mean what good are these cold sql boxes etc if I have no means to authenticate to them right?
Do I Install from media on a box at the site with the latest system states?
I am very interested to hear what some of you are doing for this. I am looking for DR or BCP whatever the money is calling it these days. I am speaking to the scope that Katrina II has hit and we need to run on DR for a year ! I am in Jville florida so this is something that very well could happen . I used 2000+ enviorement in hopes that the solutions will be to that level .
Cheers,
David
|
|
|
joe_elway
Honorable But Hopeless Addict
Ireland
6733 Posts
Status: offline |
Posted - 07/26/2006 : 04:10:08 AM
|
Dave,
Normally I'd say, treat it as an AD site an have live replication going on... I'm a very lazy person and like DR to be as simple as possible. We used to run a soft test every quarter or so to go through drills. We had to have the HQ network up and running with PC's deployed in 4 hours and we could do it in around 2.
For file servers I'd make use of R2. DFS Namespace and DFS R used together will simplify anything to do with UNC paths, i.e. any path used in live site will be exactly the same in the DR site.
But you can't have a live system.
There's two conditions to watch for where DR can be invoked: (a) You lose your primary site and every thing is dead. (b) You lose access to your primary site but eberything is alive.
A is the one to worry about. B, however, would be a bad time to recover a DC from system state if connectivity to the live site is still up. IN B's case, I'd have a Windows backup of the system state and do a DCpromo /ADV to create a new DC.
A: I guess a cold install and recovery of the system state. I'd be worried about having different h/w in the DR site. Maybe the above method for B is the way to go and then sieze the FSMO roles. It's possibly a bit messy. My motto for DR is taken from the Gridiron world: My best gameday will only be as good as my worst practice. I expect that day to be truly a mess and want things to be simple and clean.
However, there might be another possibility. You have ESX server. How are you replicating/recovering the virtual machines? I'd see this as a cool oppurtunity. I would have virtualised DC's (with the FSMO's) and recover them first. I could then recover the rest of the network.
One tip from my boss at the last job: make special plans for directors. You do not want them anywhere near you when you invoke. They'll pester you so much that they'll hamper your efforts. Have them redirected to somewhere else and give them access to the network via Citrix SSL.
Your backup media? Where is it stored? It's possibly no good to you if it's near your office (chemical leaks, power cuts, restricted travel, etc). Ideally, it should be stored securely near your DR site. The latest backups should be immediately sent offsite. |
Aidan Finn
MCSE, MVP (Virtual Machine: Systems Administration)
IT Blog: http://www.aidanfinn.com
My Photography: http://www.aidanfinnphoto.com/
My Hyper-V Book: Mastering Hyper-V Deployment
Twitter: http://twitter.com/joe_elway |
 |
|
|
jaxdave
Honorable But Hopeless Addict
USA
2212 Posts
Status: offline |
Posted - 07/26/2006 : 06:42:48 AM
|
All very good points. I do currently have my dc's at corp virtualized under ESX , most of my servers at corp are virtual for that matter, so that is the route that I am thinking on taking to accompish this task. I am in process of reviewing legato or any other backup solutions. Veritas BackupExec is not cutting it for anymore. Any suggestions about backup softwares would be graciously accepted as well. My first thought is to have Esx vi3 standard ( not on vi3 yet but will be going to it soon )some beefed up Poweredges out at the DR. I could use visioncore,legato, or good ole acronis to backup up my DC with the FSMO roles. Restore that at Dr if need be. This project is so massive in scope that I am a little overwhelmed. With some advice from my forum friends I am sure I will get my head wrapped around it and complete the task.
Cheers,
Dave |
Edited by - jaxdave on 07/26/2006 06:44:06 AM |
|
|
|
joe_elway
Honorable But Hopeless Addict
Ireland
6733 Posts
Status: offline |
Posted - 07/26/2006 : 07:36:52 AM
|
Legato (actually called EMC now) Replistor is pretty good. I'm not so sure about the EMC clustering stuff. It has apparently changed for the better since EMC took over... one of my colleagure recently did a deployment and it went well.
Backup.... I like CommVault. Dell are resellers. They have A LOT of agents and some nice archival and regulatory solutions.
Treat the DR project as a series of mini projects. Do one thing/appliction at a time. Otherwise you will get overwhelmed. It's way too easy to jump from problem to problem and not resolve anything. Concentrate on the DC's and network first. Nothing will work without them.
Going virtual for the DR makes sense... you save on h/w and you save on rack space (which I assume you pay for). I'd put in some decent servers (blades if the budget goes that far) and a SAN for the VM's. Otherwise, some decent storage type servers if the budget won't stretch.
For backing up/recovering VM's... http://www.commvault.com/pdf/CV_SolutionBrief_VMWare.pdf. You could potentially carry your entire environment from the live site to DR in a few tapes and recover very quickly. That's assuming your live site is a virtualised.
I'd deffo look at moving FSMO to the virtual DC's in live site. Then I'd recover them in DR on ESX. Your AD will be up and running in no time (you'll have to look at meta data cleanup, etc).
If at all possible though, I'd like to have some sort of link between DR and live (even a DSL VPN) and run some virtual DC's out there with their own site. They'd run DNS, WINS (if used) and DHCP. Then I'd have no AD recovery of any kind to perform. I could concentrate all of my efforts on the other services. |
Aidan Finn
MCSE, MVP (Virtual Machine: Systems Administration)
IT Blog: http://www.aidanfinn.com
My Photography: http://www.aidanfinnphoto.com/
My Hyper-V Book: Mastering Hyper-V Deployment
Twitter: http://twitter.com/joe_elway |
|
|
|
jaxdave
Honorable But Hopeless Addict
USA
2212 Posts
Status: offline |
Posted - 07/26/2006 : 08:23:08 AM
|
| This is really great advice Aiden. I really do appreciate it. Wondering ? How would you stop your clients from authenticating to the DR dc if it were Live all the time ? |
|
|
|
JSCLMEDAVE
Administrator
USA
4818 Posts
Status: offline |
Posted - 07/26/2006 : 08:33:40 AM
|
For a SMALL shop like ours... We are using a laptop with a Virtual DC loaded. It is brought in each morning and linked back up to the network. I have monitored users authenticating to it daily without issue. I have made full images of it, disconnected it from the domain and forced FSMO seizure, then had one of the domain PCs with a test domain account authenticate to it - offline network of course. I then put it back to its original state and pressed on.
As far as backup for SQL and file servers, we are using tape right now thats stored off site daily, but we are going to Firewire external storage units that are updated after the tape each day, that are also kept off site daily. So we pretty much have our SMALL network in a can each night. I know this has many "what ifs" and "this is not the best way to do it", but with our very limited budget, its been working okay.
The only big pitfall was a mix of Symantec and Admin error. But that is being fixed today... |
Tim-
"Will the woman who left her 9 kids at Wrigley field please come and claim them? - they're beating the Cubs 5-0"
|
|
|
|
deason
Old Timer
United Kingdom
343 Posts
Status: offline |
Posted - 07/28/2006 : 3:50:02 PM
|
I have to implement a full DR solution ready for my LOB support teams to install there applications by November.
Unfortunatly my corporation do not know of the benefits of VMware so its my job to sell to them that the applications will run on them, this includes applications like SAP, SAGE, Oracle DBs, SQL DBs etc.
Once I have this up and running the next job is to scale it into a SAN solution with expansion in mind.
As for DR and AD you will have to freeze your FSMO roles if you want to accomplish this correctly i think, this is standard MS practice.
VMware also provides me with the option to put 20 XP boxes on one machine for Thin Client access, I have a lot of small sites that in the event of a fire or natural disaster would be crippled with PC lead times etc from vendors. |
SMS - System Management Slave
---------- Blog @ http://vmlover.blogspot.com ------------- |
|
|
|
joe_elway
Honorable But Hopeless Addict
Ireland
6733 Posts
Status: offline |
Posted - 07/28/2006 : 5:43:27 PM
|
quote:
Originally posted by jaxdave
This is really great advice Aiden. I really do appreciate it. Wondering ? How would you stop your clients from authenticating to the DR dc if it were Live all the time ?
It's a different AD site with it's own site and subnet definitions. That's a natural boundry right there. If you are supernetting, i.e. your DR has the same network address and the live site, then it's probably not an issue because you have lots of bandwidth. |
Aidan Finn
MCSE, MVP (Virtual Machine: Systems Administration)
IT Blog: http://www.aidanfinn.com
My Photography: http://www.aidanfinnphoto.com/
My Hyper-V Book: Mastering Hyper-V Deployment
Twitter: http://twitter.com/joe_elway |
|
|
|
joe_elway
Honorable But Hopeless Addict
Ireland
6733 Posts
Status: offline |
Posted - 07/28/2006 : 5:59:34 PM
|
Back in early 05 I did some maths on virtualising a small organistion's DR. The live site had 4 DL380's with tape backup. We looked at giving them 1 server, e.g. a DL360 with cheap scsi attached disk tray or a DL380 with lots of disk and running VS 2005. Virtualising their DR for them only would have save a few hundred bucks so they went physical. Had they been running VM's in live, a virtual DR would have been a sinch. Seeing as VM tech is free now, they might have gone VM's if making the decision today.
It really all depends on what you can afford, what tech you have in live, what changes you can make to live and how fast you need to be up and running. Once you have a big picture, start dealing with the whole thing, one service/application at a time.
Ideally, I'd want a nice fat line between live and DR. I'd use ESX as much as possible and have all my FSMO's on there. I'd replicate those VM's some way out to the DR site. File servers are easy: R2 DFS namespace and DFS-R. SQL or Exchange: If I have h/w SAN replication then I go with Windows clustering. If I need s/w replication then I look at EMC Replistor and EMC Autostart. For excahnge, I'd be tempted to have a look at the continuity service from Exchange hosted Services too. More R2: use the print management console to deploy pritners.... automnate the time consuming crap.
PC's in DR: some companies rent as required. If so then I use RIS/GPO to install them or SMS with Zero Touch (os deployment and advertisements for s/w deployment). If I own the PC's in DR then they are live and turned on so AV can be managed and s/w can be managed. With SCOM 2007 I can even remotely monitor them to see if one is dead - best to fix it now than invoke the DR and find it's dead then.
It's not a bad idea to have a VPN solution or SSL terminal services/citrix in the DR. You're going to have limited seats and may need more bodies to gain access to the network.
The hardest part is owning it once you implement it. Everything you do in live should be done in DR. This costs effort and money. Bosses won't like it. This is why you need someone who owns the Business Continuity Plan, ideally a director. They can drive it. In the end you only provide the nuts and bolts. The business is going to have to figure out who goes to DR when invoked, how to invoke, how to communicate the invocation and how to get there.
BTW, in my last job our NY office put their DR in Jersey. They had a smart idea. It was a living, breathing place that not only would be used for DR but would also be used for training and for meetings when they didn't want to go to the office to avoid being disturbed. |
Aidan Finn
MCSE, MVP (Virtual Machine: Systems Administration)
IT Blog: http://www.aidanfinn.com
My Photography: http://www.aidanfinnphoto.com/
My Hyper-V Book: Mastering Hyper-V Deployment
Twitter: http://twitter.com/joe_elway |
|
|
|
deason
Old Timer
United Kingdom
343 Posts
Status: offline |
Posted - 07/30/2006 : 4:13:19 PM
|
Joe,
I think for the next 18 months DR on VMware ESX for me will be the way to go, im going to size up storage space once i get the requirements and then spec the smallest possible HP EVA to do Continuos access between sites (when we get dark fibre in).
I am also looking at Platespin this weekend and it looks very good, it replicates live services for you something that maybe handy, not quite sure how it would do this with SQL etc maybe better to use Log SHipping for that.
Could be usefull for an Exchange MDB though.
|
SMS - System Management Slave
---------- Blog @ http://vmlover.blogspot.com ------------- |
|
|
|
joe_elway
Honorable But Hopeless Addict
Ireland
6733 Posts
Status: offline |
Posted - 07/30/2006 : 6:35:23 PM
|
Replistor is excelent at replicating databases such as Exchange and SQL. It uses a filter to capture or inject file system writes and works on block level. From what I'm told Autostart has a replication mechanism of it's own built in, probably the same as Replistor. I know Autostart can use and control the Replistor specifications. I really must catch up on it. Autostart has built in definitions for controlling the exchange/sql servbices for a cluster and if it's like its predecessor, you can custom write your own for other apps.
There's a shedload of companies that provide these replication and clustering solutions.
ESX is the way to go. If I was planning a data center all over again, I'd use it. Blades, EMC SAN and ESX. Only in the rarest situations would I not use VM's. This architecture is just so h/w independant it is perfect for DR. No worrying about geo-clusters... just replicate/recover the VM's to another set of servers on a similarly addressed network and Bob's your uncle.
Just found this: http://www.virtual-strategy.com/article/view/1591/
I've not worked with ESX yet (later this year) but I guess h/w based SAN replication will also work for replicating ESX VM's to a DR site. Wouldn't mind hearing what Nathan reckons of this. |
Aidan Finn
MCSE, MVP (Virtual Machine: Systems Administration)
IT Blog: http://www.aidanfinn.com
My Photography: http://www.aidanfinnphoto.com/
My Hyper-V Book: Mastering Hyper-V Deployment
Twitter: http://twitter.com/joe_elway |
|
|
|
jaxdave
Honorable But Hopeless Addict
USA
2212 Posts
Status: offline |
Posted - 07/30/2006 : 7:58:58 PM
|
I have 6 esx servers (currently looking at vi3 upgrade in a serious way) there are some incredible features that have been added. HA,DRS,Live Hard disk adds , snapshots ( just like in vmeare workstation ) just to name a few. I am going the emc san, vmware esx, with replistor for my sql,exchange databases. I appreciate all the advice you gave me in this thread. Thanks again.
Cheers,
Dave |
|
|
|
clarinathan
Moderator
United Kingdom
4782 Posts
Status: offline |
Posted - 08/05/2006 : 03:25:34 AM
|
quote:
snapshots ( just like in vmeare workstation )
Didn't know about that one.
That is really great!
Just for infos sake, the Falconstore products for SAN replication are pretty good too and they are storage indepentent. |
Nathan Winters - MVP Exchange Server
MCSE & MCSA 2000 & 2003 + Messaging, MCITP Exchange 2007, MCP, VMWare VCP v2 & v3.
Checkout the Messaging and Mobility User Group: http://www.mmmug.co.uk
Checkout my blog:
Unified Comms: - http://www.nathanwinters.co.uk |
|
|
|
jaxdave
Honorable But Hopeless Addict
USA
2212 Posts
Status: offline |
Posted - 08/06/2006 : 1:06:39 PM
|
| Yeah, the famed snapshot technology that we know and love is in the VI3 ! It plays a big part in the Vitual Consolidated Backup ( VCB ) which is new to VI3. Another really great feature is we can now hot add a hard disk to our VMs in VI3. Memory and CPU are still a shutdown , add the hardware , power on process though. |
Edited by - jaxdave on 08/10/2006 9:23:46 PM |
|
|
|
clarinathan
Moderator
United Kingdom
4782 Posts
Status: offline |
Posted - 08/10/2006 : 4:00:47 PM
|
| Sounds great. As soon as I get ESX 3 to work with a "normal" iSCSI target I will get to try some more of these great features. |
Nathan Winters - MVP Exchange Server
MCSE & MCSA 2000 & 2003 + Messaging, MCITP Exchange 2007, MCP, VMWare VCP v2 & v3.
Checkout the Messaging and Mobility User Group: http://www.mmmug.co.uk
Checkout my blog:
Unified Comms: - http://www.nathanwinters.co.uk |
|
|
| |
Topic |
|
DR for 2000 + user enviorement
