BEMS Disaster Recovery Plan
1. Objectives of the Disaster Recovery Plan
- Enable the facilities team to act as the primary point of contact for identifying and reporting issues.
- Minimize system downtime by ensuring effective collaboration between the facilities team, BEMS contractor, electrical contractor, and IT engineers.
- Safeguard critical building operations and ensure quick recovery of the BEMS.
2. Key Elements of the Disaster Recovery Plan
2.1 Risk Assessment
Focus on the risks most relevant to the BEMS, considering the facilities team’s oversight.
1. Physical Risks:
- Power outages or fluctuations impacting BEMS control panels.
- Hardware failures of controllers, sensors, or actuators supplied by the BEMS contractor.
2. Cybersecurity Risks (if remotely accessible):
- Unauthorised access or malware affecting the BEMS contractor’s remote access platform.
3. Operational Risks:
- Human error during routine maintenance or configuration.
- Network isolation or communication issues within the BEMS silo.
2.2 Roles and Responsibilities
The facilities team takes a central role in disaster management, supported by key stakeholders.
- Facilities Team: Monitor system performance, identify and report issues, and coordinate with relevant stakeholders.
- BEMS Contractor: Diagnose and resolve BEMS issues, restore configurations, and address remote access or cybersecurity problems.
- Electrical Contractor: Restore power to the BEMS control panels and ensure voltage stability.
- Client IT Engineers: Resolve network issues if the BEMS resides on the client’s internal network.
- BEMS Contractor IT Team: Address cybersecurity incidents for systems with remote access.
2.3 Backup Strategy
Ensure a reliable backup process to minimize data loss and simplify recovery.
1. Backup Components:
- Controller Configurations: Includes control logic, sequences of operation, and schedules.
- Device Settings: Sensor, actuator, and I/O point calibration and configuration.
- Historical Data: Trend logs, alarms, and energy monitoring data, if critical.
2. Backup Frequency:
- Daily incremental backups for essential configurations.
- Weekly full-system backups, including historical data.
3. Storage Locations:
- On-Site: Secure local storage within the BEMS control panel or server.
- Off-Site: Cloud-based or remote server backups managed by the BEMS contractor.
4. Backup Testing:
- Regularly test the restoration process to ensure data integrity and recovery speed.
2.4 Incident Identification and Reporting
The facilities team is the first line of defense in identifying and reporting incidents.
1. Identification:
- Use BEMS trend logs, alarms, and fault notifications to detect anomalies or system failures.
- Conduct visual inspections of control panels and connected devices during suspected failures.
2. Reporting:
Log the incident with detailed descriptions of the issue, including:
- Affected systems or components.
- Observed alarms or error messages.
- Recent changes or events that may have triggered the issue.
- Notify the BEMS contractor immediately, providing the incident log for context.
3. Escalation:
- Engage the electrical contractor for power-related issues.
- In cases of network or cybersecurity incidents, coordinate with the client’s IT engineers or the BEMS contractor’s IT team.
2.5 Incident Response and Recovery
Facilities teams coordinate the disaster recovery process, supported by technical experts.
1. Immediate Response:
- Isolate affected systems to prevent further damage (e.g., disable faulty controllers or devices).
- Activate manual overrides for critical systems (e.g., HVAC or safety systems) to maintain building operations temporarily.
2. Power and Hardware Restoration:
- Electrical contractor verifies power supply integrity and restores power to control panels.
- BEMS contractor replaces faulty hardware and restores functionality using spares.
3. Configuration Restoration:
- BEMS contractor deploys the most recent backup to recover control logic and device settings.
- Validate communication between controllers, sensors, and actuators.
4. System Testing:
- Conduct functional tests to verify restored sequences of operation.
- Ensure alarms, schedules, and reporting features are fully operational.
5. Cybersecurity Response (if applicable):
- If remote access is compromised, the BEMS contractor escalates to their IT engineers to:
- Investigate the breach and secure the system.
- Restore remote access functionality with enhanced security measures.
2.6 Documentation and Communication
Facilities teams manage incident records and keep stakeholders informed throughout the recovery process.
1. Incident Logs:
- Maintain a detailed log of the incident, including detection, response actions, and resolution.
2. Stakeholder Updates:
- Provide regular updates to building owners, management teams, and other stakeholders.
3. Post-Recovery Report:
Prepare a comprehensive report summarizing:
- The root cause of the incident.
- Actions taken during recovery.
- Recommendations to prevent future occurrences.
3. Preventative Measures
To reduce the likelihood of disasters and improve recovery times, implement proactive strategies.
1. Facilities Team Training:
- Train facilities staff to identify early warning signs (e.g., abnormal trends, alarms) and escalate appropriately.
2. Routine Maintenance:
- Conduct regular inspections and servicing of hardware components, including controllers and sensors.
3. Power Resilience:
- Install uninterruptible power supplies (UPS) or backup generators for critical systems.
4. Cybersecurity Enhancements:
- Use firewalls, VPNs, and multi-factor authentication for remote access.
- Conduct regular cybersecurity audits to identify vulnerabilities.
5. Backup Testing:
- Regularly test the restoration process using the latest backups to ensure readiness.
4. Deliverables
- Disaster Recovery Plan Document: Comprehensive plan detailing roles, responsibilities, and recovery steps.
- Incident Logs: Detailed records of detected incidents, including actions taken and outcomes.
- Backup Logs and Schedules: Documented records of backups, including frequency and storage locations.
- Post-Recovery Reports: Summaries of incidents, root causes, and recommendations for future prevention.
Conclusion
This disaster recovery plan emphasizes the facilities team’s central role in identifying and managing incidents, coordinating with technical experts, and ensuring a smooth recovery process.
By integrating proactive strategies, effective communication, and structured recovery procedures, the plan ensures that the BEMS remains resilient to disruptions while safeguarding critical building operations.