Senior Software Engineer, ML Evaluation

Mission:

We are seeking a highly skilled and motivated Senior Data Analysis Engineer for our large-scale AI model and software evaluation framework – Ground Truth Regression. The ideal candidate will have a strong background in data engineering, machine learning principals and statistical analysis.

At Motional, large scale orchestration for data analysis plays a critical role in delivering our ML-centered autonomous driving vehicle. Our robo-taxi ML Models and software are developed by hundreds of developers and deployed multiple times a day – The Ground Truth Regression team provides the large scale perception analysis framework for validating all changes to perception which can impact the autonomous vehicle behavior.

The GTRegression team validates the end impact of ML and software changes to the autonomous vehicle while those changes are in development, providing the tools for Root Cause Analysis, and safety consideration. We monitor for model errors, anomalies, rare objects & long-tail driving scenarios across thousands of driving hours. The team develops the full stack – AWS Kubernetes orchestration framework, ReSimulation, model metrics, regression reports, and deep dive analysis UI’s. The team works together with all other perception and ML teams to develop the metrics and analyze results.

What You'll Do:

• Work with ML Engineers and Autonomy Software Developers to develop new data analysis metrics and KPI’s for the validation of the autonomous vehicle performance.
• Drive innovation by researching and developing new large scale data analysis systems.
• Own large-scale data analysis workflows that surface unexpected impacts of planned and unplanned changes to the ML models and software.
• Work with annotation specialists to build high-quality evaluation datasets.
• Provide statistical depth on model performance & generalization through rigorous error and change analysis.
• Read, understand, and modify complex codebases written in Python and SQL.

What We're Looking For:

…