Content + AI = Knowledge

Your content is a source of valuable knowledge. But it is buried because it may not be in a form that users or new AI capabilities can consume or interact with. The digital form of old documents is not convenient to read or search. This problem is clear when looking at documents digitized on Google Books — they’re hard to read and only offer basic text search. However, using AI, this content can be converted into HTML format. HTML content is highly readable (on all types of devices) and can be indexed for semantic search, QA, and reasoning.

New AI capabilities can easily interpret text based content but struggles with more complex structures such as tables, schematics, jump pages, multi-column layouts, page content containing noisy blocks and even bullet points.. In this article, we would later see that the Trillo Doc AI solves this problem and makes their content available online for browsing, search, Q&A and reasoning using AI.

Google Books Page — Hard to read on mobile and only supports basic search

Your organization likely has tons of these documents: manuals, designs, old magazines, contracts, financial statements, and more. Right now, this information is only accessible to a few who already know where to find it. They waste time searching, sometimes hours, for the right document and page. AI can change this. It turns hidden content into knowledge accessible to any authorized user, allowing them to find what they need in seconds. Users can then interact with the information using AI: summarizing it, extracting key concepts, downloading data, and asking questions. The possibilities are endless.

This means users can quickly become experts on any topic. Even seasoned experts can discover new insights, like patterns between successful and problematic contracts. Imagine having all the content ever created, by humanity or just your organization, extracted and analyzed by AI. Imagine interacting with it through an AI agent, learning for pleasure or profit.

Think about that 900-page bill from Congress that is impossible to understand. With AI, you could interactively query it and understand its implications. Or imagine learning from aircraft maintenance manuals, taking notes, and discussing them with an expert, all aided by AI.

Combining AI with human intelligence allows for powerful learning grounded in reality. It lightens the cognitive load and makes the process more enjoyable. The best part? Platforms are available right now to help you get started in minutes.

Example Use Cases

Before you jump into a platform, let’s explore a few use cases to spark your imagination.

Old Magazines and Periodicals: Imagine you’re a publisher with a treasure trove of content, some of it going back to the 1800s. Much of this valuable material might be gathering dust in physical archives or buried in scanned files. Bringing these issues online would be a huge benefit to your readers. Think of a literary magazine like Kadambini (published in India until 2020) — making its entire archive available online would delight readers, and they would be happy to pay for access.

Industrial Manuals: Think about aircraft maintenance manuals, oil rig operation guides, tool specifications, engineering designs, and similar documents. AI can analyze, format, and catalog all of this, making it easily searchable. Users could query these manuals using text, images, charts, or schematics.

Office Documents: AI can be incredibly useful for everyday office documents like contracts, resumes, purchase orders, bank statements, mortgages, and marketing materials. It can power advanced document search, extract key data for workflow automation, and perform semantic matching. For example, AI can match job descriptions to resumes or analyze contracts for specific clauses. And, of course, it can search across all these document types.

Content Processing Platform or Knowledge Agent

So, what platforms can convert your content into usable knowledge?

We’ll focus on Google Cloud AI platforms and our own platform, Trillo Doc AI. We’re including Trillo Doc AI because we believe its features are particularly useful for complex documents that require customization. While other companies and hyperscalers (like Azure and AWS) offer similar agent builder tools and services, we won’t discuss them here due to our limited knowledge of their specific functionalities.

Google Cloud NotebookLM: This platform is incredibly user-friendly. As their website says, “Upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides, and NotebookLM will summarize them and make interesting connections between topics, all powered by Gemini 2.0’s multimodal understanding capabilities.” It’s designed to help you understand complex information by providing a single place for reading, thinking, asking questions, and writing.

Vertex AI Agent Builder (Google Cloud): This platform allows you to build AI agents, from no-code solutions to highly customized ones. Think customer service chatbots, employee agents that automate tasks, knowledge agents for search and Q&A, and even voice agents. For extracting knowledge, the knowledge agent is most relevant. The basic idea is similar to NotebookLM: upload documents, process them, and then enable searching and question answering. Vertex AI Agent Builder also offers connectors to enterprise applications for ingesting content and data. Developers can also create custom or specialized agents for additional needs.

Google AgentSpace: Google AgentSpace is delivered as a SaaS application (like Google Workspace) and built on the top of Vertex AI Agent Builder. It is simpler to use. It offers a more limited set of developer tools and is fully managed by Google Cloud. Vertex AI Agent Builder, on the other hand, is a more comprehensive platform for building and managing AI agents.

Trillo Doc AI: We’ll dedicate the next chapter to Trillo Doc AI, explaining why and how it provides an extended knowledge agent using Gemini and vector databases available on Google Cloud.

Trillo Doc AI

When you look closely, all knowledge agent platforms essentially do the same thing: they ingest content from various sources, process it using a large language model (like Gemini), and then make it available for searching, Q&A, and reasoning. For many common office documents, the processing pipeline is standard. However, documents with specialized domains, layouts, and content require a custom approach. Think about these examples:

1. Old, multi-column magazines with photos, tables, paragraphs, and other elements.
2. Engineering documents like operation manuals, designs, and customer service content, which often include tables, schematics, drawings, and charts.
3. Academic papers with formulas, tables, charts, and equations.

Essentially, any document can benefit from a custom processing approach.

Beyond just processing documents for search, Q&A, and reasoning, Trillo Doc AI also extracts content in a structured format. This means it identifies and extracts headings, paragraphs, bullet points, tables, images, charts, and more. This structured data can then be used to display the document content clearly on any device, ensuring high readability. It’s also crucial for building semantic indexes.

Trillo Doc AI is a platform for building custom pipelines with no-code or low-code (using prompt engineering and serverless functions). In addition, Trillo Doc AI extracts structured parts of documents for intelligent processing and user friendly rendering.

Trillo Doc AI — Example Pipeline for Processing Magazine

Here’s an example of a processing pipeline implemented by Trillo Doc AI for magazines (or any bound document, like a book). It follows these steps:

1. Extract Content: Trillo Doc AI extracts the content using OCR or open-source libraries. This pulls out the text and images embedded in the document. It also generates preview images and thumbnails at this stage.
2. Clean Content: Trillo Doc AI then converts the page content into structured content (JSON format)
3. Summarize: Using the cleaned content, Trillo Doc AI creates a summary of the document.
4. Image Properties: Trillo Doc AI extracts image properties using AI Vision and/or Generative AI.
5. Extract Tags: Trillo Doc AI extracts tags to enable filtering and searching by categories.
6. Extract TOC: The table of contents is extracted by Trillo Doc AI.
7. Make Chapters: For easier browsing, Trillo Doc AI constructs chapters.
8. Clean Chapters: When pages are combined into chapters, Trillo Doc AI cleans them up. This might involve things like combining paragraphs that were split across pages.
9. Post Processing: Finally, Trillo Doc AI uses serverless functions to perform any other rule-based processing on the content.

Trillo Doc AI Pipeline to process documents like magazines

Trillo Doc AI — Example Pipeline for Extracting Data from Variety of Documents

Trillo Doc AI supports creating custom workflows for different needs. For example, if there’s a need to extract common fields (say, for asset management) from a variety of documents, the following workflow can be used:

1. Extract Content: Trillo Doc AI extracts content from the document using OCR or open-source libraries. It extracts text and images embedded in the document and prepares preview images and thumbnails.
2. Classify Document: Trillo Doc AI classifies the document using Generative AI.
3. Extract Data: Trillo Doc AI extracts data using Generative AI, other NLP processors, and proprietary algorithms.
4. Post Process: Trillo Doc AI post-processes the content for validation, storage, and other needs.

Trillo Doc AI pipeline to process of variety of documents to extract data

Conclusion — A Summary of Trillo Doc AI Features

In short, Trillo Doc AI offers a powerful set of features crucial for effective content processing.

1. Trillo Doc AI leverages the power of Gemini and other Large Language Models. Its API-driven design also allows it to seamlessly integrate with other platforms, like NotebookLM Enterprise, extending their capabilities.
2. Trillo Doc AI excels at custom document processing, a critical need for complex, domain-specific documents.
3. It extracts content elements — headers, paragraphs, tables, bullet points, charts, images, schematics, graphs, and more — as structured JSON data. This allows for the creation of highly readable content across all devices, dramatically improving user experience.
4. Recognizing that AI isn’t always perfect, Trillo Doc AI incorporates tools for manual review and content curation, putting humans in the loop for essential quality control.
5. Trillo Doc AI provides access control that is easy to define, audit and performs well at search scale.
6. Finally, Trillo Doc AI is built on Trillo Workbench, a robust low-code platform. Trillo Workbench provides a model-driven architecture for data, serverless functions for logic, and a high-level API that abstracts away the complexities of the cloud. This foundation allows us to rapidly add new steps to the processing pipeline without the headaches of security, scalability, compliance, or CI/CD.
7. You can find more about Trillo Doc AI here.