How to Approach ISP Tuning for ML
- Timofey Uvarov
- Jan 14
- 1 min read
Updated: Jan 23
Tuning is a continuous process, so it is essential to start with a specification and finish it when the specification is met. The specification is compiled by modulating optical benchmarking results with perception requirements. The first reference point is created by benchmarking available solutions. The following specification is constructed by translating perceptual requirements into optical and HW specs.
Perceptual requirements are collected by researching vision trends and categorization of camera issues.
In robotic and OEM companies, issues are categorized by frequency and degree of malfunctioning of the agent, and in applications targeted for human vision - by surveying the target group.
Perception requirements can be as simple as removing color tint or more advanced, like detecting a specific moving object at a certain distance, speed, and illumination. In the spec, it's important to register all operational states and classify whether the camera is optimized for detecting only certain types of objects or needs to treat everything we observe in the scene equally. We can dynamically change the configuration if the agent knows its state and target and can communicate it to the ISP.
Tuning hyperloop cycles through iterations of benchmarking and tuning sessions unless the performance spec is met.
Examples of tuning
