Business Continuity and Disaster Recovery (BC/DR)
Policy Owner: Jarvis (Hai) Luong
Effective Date: 01.03.2025s
Purpose
The purpose of this business continuity plan is to prepare Tekai in the event of service outages caused by factors beyond our control (e.g., natural disasters, man-made events), and to restore services to the widest extent possible in a minimum time frame.
Scope
All Tekai IT systems that are business critical. This policy applies to all employees of Tekai and to all relevant external parties, including but not limited to Tekai consultants and contractors.
The following scenarios are excluded from the BC/DR plan scope: Loss of availability for a production hosting service provider
Loss of availability of Tekai satellite offices (these will be considered incidents)
In the event of a loss of availability of a hosting service provider, the CIO/CTO will confer with the Technical Lead to determine an appropriate response strategy.
Policy
In the event of a major disruption to production services and a disaster affecting the availability and/or security of the Tekai office, senior managers and executive staff shall determine mitigation actions.
A disaster recovery test, including a test of backup restoration processes, shall be performed on an annual basis.
Continuity of information security shall be considered along with operational continuity.
In the case of an information security event or incident, refer to the Incident Response Plan.
Alternate Work Facilities
If the Tekai office becomes unavailable due to a disaster, all staff shall work remotely from their homes or any safe location.
Communications and Escalation
Executive staff and senior managers should be notified of any disaster affecting Tekai facilities or operations.
Communications shall take place over any available regular channels including Slack, email, phone and Google Meets.
Key contacts shall be maintained on the on-call schedule and key contacts:
- CIO: Jarvis Luong (jarvis@tekai.fi | 0505519989)
- CTO: Hieu Nguyen (hieu@tekai.fi | 0458465768)
Roles and Responsibilities
Role | Responsibility |
---|---|
Technical Lead | The Technical Lead shall lead BC/DR efforts to mitigate losses and recover the corporate network and information systems. The Technical Lead shall be responsible for leading efforts to maintain continuity of Tekai services to customers during a disaster. |
Managers | Managers shall be responsible for communicating with their direct reports and providing any needed assistance for staff to continue working from alternative locations. |
CIO | The CIO, in conjunction with the CEO and COO shall be responsible for any external and client communications regarding any disaster or business continuity actions that are relevant to customers and third parties. |
COO | The COO shall be responsible for internal communications to employees as well as any action needed to maintain physical health and safety of the workforce. The COO shall work with the Lead Engineer to ensure continuity of physical security at the Tekai office. |
Continuity of Critical Services
Procedures for maintaining continuity of critical services in a disaster can be found in Appendix A. Recovery Time Objectives (RTO) and Recovery Point Objects (RPO) can be found in Appendix B. Strategy for maintaining continuity of services can be seen in the following table:
KEY BUSINESS PROCESS | CONTINUITY STRATEGY |
---|---|
Customer (Production) Service Delivery | Rely on service provider availability commitments and SLAs |
IT Operations | Not dependent on HQ. Critical data is backed up to alternate locations (Github | Gitlab | Bitbucket). |
Utilize Gmail and its distributed nature, rely on Google's standard service level agreements. | |
Finance, Legal and HR | All systems are vendor-hosted SaaS applications. |
Sales and Marketing | All systems are vendor-hosted SaaS applications. |
Plan Activation
This BC/DR shall be automatically activated in the event of the loss or unavailability of the Tekai office, or a natural disaster (i.e.,
severe weather, regional power outage, earthquake) affecting the larger Helsinki region.
Version | Date | Description | Author | Approved by |
1.0 | 01.03.2025 | First version | Lucas | |
Appendix A - Business Continuity Procedures by Scenario
Business Continuity Scenarios
HQ Offline (power and/or network)
CRM, Telephony, Video Conferencing/Screen Share & Corp Email unaffected SUPPORT unaffected
HQ Staff offline (30-60 minutes) Remote Staff unaffected (US)
Procedure:
- HQ Staff relocate to home offices (30-60 minutes)
- Verify Telephony, CRM, & Email Connectivity at home offices (10 minutes)
- Remotely resume normal operations
Colo Offline (power and/or network)
CRM, Telephony, Video Conferencing/Screen Share & Corp Email unaffected SUPPORT Offline
Production Database offline (redundant) HQ Staff unaffected
Remote Staff unaffected (US)
Procedure:
- Notify Customer Base that proactive monitoring is offline
- Normal operations continue
Disaster Event at HQ Helsinki
CRM, Telephony, Video Conferencing/Screen Share & Corp Email unaffected SUPPORT offline
HQ Staff offline (variable impact) Remote Staff unaffected (US)
Procedure:
- Activate Remote Staff (US)
- Notify Customer Base of impaired functions & potential delays
- Commandeer Field Resources for Critical Response (SE Teams)
SaaS Tools Down
CRM, Telephony, Video Conferencing/Screen Share, or Corp Email Affected
SUPPORT partially affected (no new cases, manual triage required) HQ Staff unaffected
Remote Staff unaffected (US)
Procedures:
Telephony Down
- Notify Customer Base to use Support Portal or Email
- Support Staff use Mobile Phones and/or Land Lines as needed Email Down (Gmail/Corp Email)
- Support Staff manually manage ‘case' related communications
- Support Staff use alternate email accounts as needed (Hotmail) CRM Down
- Notify Customer Base that CRM is down
- Activate ‘Spreadsheet' Case Tracking (Google Sheets)
- Leverage ‘Production' Database for Entitlements, Case History, Configuration data. Video Conferencing/ScreenShare Down (Zoom)
- Support Staff utilize alternate service as needed
Appendix B - RTOs/RPOs
Rank | Asset | Affected Assets | Business Impact | Users | Owners | Recovery Time Objective (RTO) | Recovery Point Objective (RPO) | Comments / Gaps |
1 | Google Datacenters | Site | Core services | All | Engineering | |||
Google Cloud | Network | Core services | All | Engineering | ||||
Home Office ISP Networks | Network | IT Ops, Development | N/A | |||||
Subcontractor Networks | Network | Development | N/A | |||||
Third Party Networks | Network | Sales | N/A | |||||
Company Laptops | Hardware | All | IT Ops | |||||
Digital Projector | Hardware | All | IT Ops | |||||
Office Printers | Hardware | Inability to print in corporate office | All | IT Ops |
Personal Mobile Device | Hardware | |||||||
Wireless Access Points (WAP) | Hardware | All | IT Ops |