Was testing OCR on a doc-to-vector pipeline today, screenshots and images going into chunks for semantic search. Code was agent-written, and the query side just was not returning results for text pulled from certain screenshots. First thought was a logic bug in the code. Then I stopped and went straight to Tesseract's own online OCR tool to check if the problem was the package itself or my code.
Tested a screenshot with highlighted text, the blue hover-state you get on a menu item, and it failed there too, same as in my pipeline. Took a new screenshot without the highlight, and it worked fine. That is when it clicked: the highlighted or hovered state of the text was distorting what raw OCR could actually read.
Fix was adding image preprocessing before OCR, so the screenshot gets cleaned up first and Tesseract can pick up highlighted text reliably.
Whenever I get stuck, find the exact spot or the actual culprit first, whether that turns out to be a code issue, a product outage, or an AWS infra issue, then work outward from there. Left alone, an agent will often just keep refactoring the surrounding logic instead of finding that spot, which wastes time and can end up hiding the real issue even further. In this case, the culprit was never the query logic at all, it was the highlighted text breaking OCR upstream.