Search

When Things Go Wrong: Troubleshooting Bad Bettrdata Runs

Looking for Reports from A Run

This is to find if data is not being produced

  1. Open Studio 3T
  2. Connect to app instance
  3. Go to the di2 database
  4. Open the datafile collection
  5. Look for the latest data produced (assuming you just ran the job), with the most recent id
  6. From Studio 3T, in the sort text box, add the value
    1. {"$natural": -1}
  7. Copy the value of the _id as text
  8. Go to the convertrun collection and query using the id to find the file
    1. { "file._id" : "66f7085f5b31960017c8264c" }
  9. Copy the _id value as text for that record

Finding a Report

Using the record id from convertrun to see if there was a report created for it

  1. Query with the _id from the convert run to find the report
    1. { _id : ObjectId("66fc2dcb5b31960017c82661") }
  2. A report records should appear if the run is finished

Finding a Client

  1. From the convertrun record, click on the file field and use the shortcut key cntr-j. This should bring up a window with the json about the file metadata
  2. Select the value for /file/belongsTo
  3. Open the client collection
    1. Use the copied belongsTo value to find the client as an object id
    2. { _id : ObjectId("66f706555b31960017c82645") }

Finding the File on The Server

Use the convertru._id to find the files in the server

  1. Ssh to the instance
    1. ssh ec2-user@infomat.bettrdata.io
  2. Change directory to /mnt/data/input
  3. Find the file data by the id
    1. find . -name 669548575b31960017c82406
  4. Change directory to that folder
  5. Check the status of the data “parts-*” files
    1. wc -l part*
    2. head -10 part-00000