Company:
Qualcomm India Private Limited
Job Area:
Engineering Group, Engineering Group > Software Test Engineering
General Summary:
JD for Model Accuracy Development and Test Engineer
Role Overview:
We are seeking an Inference Accuracy engineer to design, develop, and validate model accuracy of deep learning models deployed at scale. The role focuses on deep accuracy analysis, debugging, accuracy evaluation, and recovery during inference on large data-centre hardware platforms. This position requires strong problem-solving ability, excellent Python programming skills, and hands-on expertise with inference pipelines.
Key Responsibilities:
- Define and implement accuracy KPIs across precision modes
- Develop scalable Python-based accuracy evaluation tools and automated pipelines.
- Implement accuracy-preserving optimizations for inference frameworks (TensorRT, ONNX Runtime, AITemplate, Triton).
- Build and maintain automated pipelines for accuracy evaluation across multiple frameworks (ONNX, TensorFlow, PyTorch).
- Develop reusable plugins for preprocessing, post-processing, and metric evaluation.
- Execute comprehensive accuracy tests for large-scale models (LLMs, vision, diffusion).
- Validate accuracy under various quantization and precision settings (FP32, FP16, INT8).
- Perform accuracy analysis with deep understanding of model architecture, including layers, attention mechanisms, and parameter configurations.
- Identify architecture-driven accuracy degradation trends and propose optimization strategies.
- Identify issues related to preprocessing drift, tokenization mismatches, operator fallback, and quantization effects.
- Analyze accuracy differences across hardware targets, firmware versions, and runtime backends.
- Perform slice-based accuracy analysis (batch size, concurrency, sequence length, domain shifts).
- Design and run experiments to recover accuracy, including fine-tuning, calibration, and hyperparameter adjustments.
- Debug accuracy failures by tracing root causes across data preprocessing, model layers, quantization steps, and deployment pipelines.
- Compare results across different hardware/software stacks and generate actionable insights.
- Document workflows, maintain dashboards, and publish accuracy results for stakeholders.
Required Skills :
- Strong background in AI/ML model evaluation and accuracy metrics.
- Solid understanding of model architectures (transformers, CNNs, RNNs, MoE) and their impact on accuracy.
- Experience with large language models (LLMs) and generative AI accuracy validation.
- Expertise with inference runtimes (TensorRT, ONNX Runtime, Triton).
- Understanding of quantization (INT8/FP8/INT4), calibration, QAT, and accuracy trade-offs.
- Experience with model graph conversion (PyTorch → ONNX → backend engines).
- Hands-on experience with accuracy pipeline development and automation frameworks. Understanding of video generation model accuracy and multi-modal evaluation benchmarking
- Proficiency in Python and familiarity with ML toolkits (ONNX Runtime, TensorFlow, PyTorch).
- Expertise in accuracy analysis, including statistical methods and visualization tools
- Ability to design experiments for accuracy recovery and debug accuracy failures effectively.
- Knowledge of quantization techniques and mixed-precision workflows.
- Experience with data-centre accelerators (NVIDIA A100/H100/B200, AI100 Ultra, Gaudi, TPU).
- Knowledge of LLM accuracy evaluation tools (lm-eval, HELM, synthetic benchmarks) is an advantage
- Strong problem-solving and analytical skills with the ability to isolate complex accuracy issues.
- Familiarity with distributed deployment systems (Kubernetes, cloud inference services).
Qualifications:
- Bachelor's / Masters degree in Engineering, Machine learning/ AI, Information Systems, Computer Science, or related field.
- 2-5 years’ of Software Engineering or related work experience.
- 2-5years’ experience with Programming Language such as C,C++, Python.
Minimum Qualifications:
• Bachelor's degree in Engineering, Information Systems, Computer Science, or related field and 2+ years of Software Test Engineering or related work experience.
OR
Master's degree in Engineering, Information Systems, Computer Science, or related field and 1+ year of Software Test Engineering or related work experience.
OR
PhD in Engineering, Information Systems, Computer Science, or related field.
• 1+ year of work or academic experience with Software Test or System Test, developing and automating test plans and/or tools (e.g., Source Code Control Systems, Continuous Integration Tools, and Bug Tracking Tools).
Applicants: Qualcomm is an equal opportunity employer. If you are an individual with a disability and need an accommodation during the application/hiring process, rest assured that Qualcomm is committed to providing an accessible process. You may e-mail disability-accomodations@qualcomm.com or call Qualcomm's toll-free number found here. Upon request, Qualcomm will provide reasonable accommodations to support individuals with disabilities to be able participate in the hiring process. Qualcomm is also committed to making our workplace accessible for individuals with disabilities. (Keep in mind that this email address is used to provide reasonable accommodations for individuals with disabilities. We will not respond here to requests for updates on applications or resume inquiries).
Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law.
To all Staffing and Recruiting Agencies: Our Careers Site is only for individuals seeking a job at Qualcomm. Staffing and recruiting agencies and individuals being represented by an agency are not authorized to use this site or to submit profiles, applications or resumes, and any such submissions will be considered unsolicited. Qualcomm does not accept unsolicited resumes or applications from agencies. Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications.
If you would like more information about this role, please contact Qualcomm Careers.
