Disaster Recovery

Date Performed	Participants	Outcome
12/07/2021	Karim Joseph	Test was completed successfully where data was intentionally corrupted and restored from the latest available snapshot.
02/09/2024	Karim Joseph & Donovan Morris	Test was completed successfully where data was intentionally corrupted and restored from the latest available snapshot. Documentation was updated for clarity and redundancy.
12/27/2024	Karim Joseph, Donovan Morris, Robert de Lorimier	- Inconsistency in what volumes are and are not backed up. However, data volumes are always backed up. - Improved documentation for use of /home/bettrdata/.bettrdata/sync-config.sh. - Full disaster recovery from scratch (creating a brand new instance). Added directions for bringing up a new instance. - Added documentation to note what drives are to be detached and attached from `lsblk` .

‣

General Documentation

‣

Find the appropriate restore snapshot and create a volume

‣

Make sure the volume is created in the appropriate region/availability zone

‣

Find and attach the volume

‣

Shutdown the BettrData platform

‣

Identify devices on the instance

‣

Note: The UUIDs should match on Current and Snapshot. If you want to double check:

‣

Unmount the Current volume

‣

Detach the Current volume

‣

Remount the volume

‣

Start all of the services again

‣

Cleanup

If it is necessary to restore a root volume. For consistency, it’s best to rebuild an environment entirely and re-attach a data volume from snapshot as outlined above.