How AstraClaw AI Reads Inside Your Files: OCR and Semantic Recognition

The most powerful capability AstraClaw AI offers — searching inside your files — depends on two foundational components working together: an OCR engine that extracts text from any file format, and a semantic engine that understands what that text means and what kind of document it came from.

The OCR engine: reading every format

OCR (Optical Character Recognition) is the process of extracting machine-readable text from images and scanned documents. Astra's OCR engine handles six different input types with tailored strategies for each.

PDFs are processed first by extracting any embedded text layer directly — this covers machine-generated PDFs like invoices, statements, and e-tickets, and is fast. When the embedded text is too short to be useful (typical of scanned PDFs), the engine renders each page as a high-resolution image and runs image OCR on it.

Images (JPG, PNG, WEBP, TIFF, and others) go through a five-step preprocessing pipeline before OCR is attempted. The raw image is processed as: original grayscale (baseline), high-contrast grayscale (2× contrast boost to sharpen faint text), sharpened and contrast-boosted (good for printed documents), adaptive binary threshold (particularly effective for handwritten text on textured paper), and inverted (for white text on dark backgrounds). All five variants are run through an OCR engine and the result with the best quality score — measured by the ratio of readable alphanumeric characters — is selected as the final output.

Word documents (.docx) have their text extracted from all paragraphs and all table cells using a dedicated document parser.

PowerPoint files (.pptx) have text extracted from every shape's text frame on every slide.

Excel files (.xlsx) have all non-empty cell values from every worksheet joined and indexed.

Google Drive documents (Docs, Sheets, Slides) are handled using Google's own full-text search API — which leverages Google's internal indexing infrastructure and requires no file download at all. This is both faster and free.

"For your scanned IRCTC ticket saved as 'IMG_0042.jpg', Astra performs: original grayscale → contrast boost → sharpened → adaptive threshold → inverted. The best OCR output wins. Your ticket becomes searchable."

Result caching

OCR is computationally meaningful — running it on every file on every search would be unacceptably slow. Astra caches all extracted text on a per-user, per-file basis, keyed by the file's last modification time. If a file hasn't changed since Astra last processed it, the cached text is used immediately. If the file has been modified, the cache is invalidated and OCR is re-run. This cache-first approach means most file content searches complete in milliseconds rather than seconds.

The semantic engine: understanding document types

Extracting text is the first step. Understanding what that text means — what kind of document it came from — is the second. Astra's semantic engine classifies every processed file into one of 15 document categories by applying pattern-based recognition against the extracted text.

The 15 categories are: Train Ticket (IRCTC bookings, PNR numbers, berth/coach information), Flight Ticket / Boarding Pass (airline, gate, terminal, seat), Bus Ticket (boarding/drop points, operator), Cab Receipt (trip fare, distance, driver rating), Hotel Booking (check-in/out dates, room type), Invoice / Bill / Receipt (amounts, tax, vendor name), Bank Statement (IFSC code, transactions, account details), ID Card (Aadhaar, PAN, Passport, Voter ID, Driving License), Medical Prescription (patient name, diagnosis, hospital), Resume / CV (experience, skills, education), Marksheet / Results (grades, roll number, institution), Insurance Policy (premium, coverage, nominee), Legal Agreement / Contract (clauses, parties, witness), Certificate (awarded to, achievement), Food Order Receipt (restaurant, delivery platform, items).

Entity extraction

Beyond classification, the semantic engine extracts specific named entities from each document type. For train tickets, this includes station codes (from a database of 70+ Indian Railway station abbreviations), journey dates, and PNR format patterns. For financial documents, it includes monetary amounts in multiple currency formats. For ID documents, it includes the 12-digit Aadhaar pattern and PAN format. For all documents, dates are extracted in multiple common formats.

Each extracted entity is added to a searchable tag cloud for that file, alongside synonyms for the document type. A train ticket gets tags including "train", "railway", "ticket", "travel", "booking", plus the full city names expanded from the detected station codes. This tag cloud is what makes searches like "find travel documents from Jaipur" work even when no file has those exact words in its name or visible text.

The combined search advantage

The combination of OCR (raw text) and semantic tagging (meaning) gives Astra two independent search paths for every processed file. A search for "electricity bill" can match via raw OCR text (if the words "electricity bill" appear literally inside a scanned document) or via semantic tags (if the document was classified as an Invoice from a utility provider). Either path is sufficient to surface the right file. Both paths together ensure very high recall — very few relevant files are missed — while the VerificationAgent's confidence scoring ensures high precision — very few irrelevant files are returned.