Why ChatGPT Is the Superior AI Tool for SEC Filing Analysis
Outperforming Competitors in Free Cash Flow Calculations and Accurate Data Extraction
In the ever-evolving world of investing, efficiency and accuracy are paramount. As part of my commitment to providing actionable insights in Safe Harbor Stocks, I’ve been exploring new tools to enhance research, specifically the use of AI chatbots to analyze SEC filings. The SEC filings contain valuable financial data, such as free cash flow (FCF), which is critical to evaluating companies like Academy Sports (ASO).
In this article, I’ll walk through my experience comparing five different AI chatbot tools:
ChatGPT (free version)
Google Gemini (free and Advanced)
The goal was for the chatbot to access and extract necessary information from the HTML or PDF versions of SEC filings to accurately calculate the free cash flow, excluding any growth-related capital expenditures.
The Challenge: Analyzing SEC Filings
The process of analyzing SEC filings is crucial for evaluating the financial health of a company. For Academy Sports, this involves parsing complex documents such as the 10-K and 10-Q filings to identify key data points, including operating cash flow, capital expenditures, and other necessary adjustments to calculate free cash flow.
This task is particularly interesting because it involves calculating a simple metric but one with various nuanced components, the information for which is located throughout the financial statements and also the Notes to the financial statements.
Data providers are notorious for providing inaccurate information and I often compile and verify this information myself. While this is a task that requires precision and attention to detail, the right tools can significantly streamline the process.
Read this article for some of the metrics I consider in building my portfolio:
Comparing the Tools
ChatGPT (Free Version)
ChatGPT, even in its free version, emerged as the superior tool for this task. Its ability to read and analyze the SEC filings was far more effective than the other tools tested. I uploaded the relevant filings in PDF form for Academy Sports, and ChatGPT quickly extracted the necessary data points, including operating cash flow and capital expenditures, to calculate free cash flow.
ChatGPT accurately excluded growth-related capital expenditures, yielding a more precise 14% FCF yield (compared to 11%). The analysis necessarily began with the 9-month period ending Q3 2024. I was impressed that ChatGPT handled three separate filings, correctly subtracting data in quarterly filings from data in the annual filing, all without issue.
The user-friendly interface allowed for seamless interactions, making it easy to ask follow-up questions and refine the analysis. ChatGPT did require a few edits but understood my instructions and easily corrected the result.
One key advantage of ChatGPT is its ability to understand and contextualize financial terminology, making it an invaluable resource for analyzing complex financial statements. This capability made it my go-to tool for research, and I plan to use it regularly going forward. Of course, it’s important to always “AI but Verify".
This chatbot even includes a link to the analysis output from its calculations in Python.
Google Gemini (Free and Advanced Versions)
Google Gemini’s free version had its drawbacks. It did not allow me to upload files, which limited its usefulness for extracting information directly from the SEC filings. While it is great for general web research, this functionality gap made it less effective for this specific task. The Advanced version (1.5 Pro), however, did allow me to upload documents, but it struggled to extract the correct information from the filings. It was not able to calculate free cash flow accurately like ChatGPT.
On a more positive note, Gemini Advanced did excel at sourcing qualitative information, such as commentary from earnings call transcripts and more general financial insights. Gemini offered use of 1.5 Pro with “Deep Research”, where it takes your prompt and outlines the tasks required. I found this functionality as good as using ChatGPT free or GPT-4o.
If you're looking for a tool to help with background research and broader company analysis, Gemini Advanced could be useful. However, when it came to crunching numbers from SEC filings, it fell short compared to ChatGPT.
TheSEC.ai
TheSEC.ai did not live up to expectations. While the platform claimed to analyze SEC filings, it failed to provide the flexibility needed for in-depth research. It seemed to be more geared toward offering pre-generated analysis, and there were no options to query the tool for specific information or adjustments in real time. Instead, the site promised that it would offer its own analysis at a later date, which was not helpful for the task at hand. In a fast-paced investing environment, this delay made TheSEC.ai less reliable as a research tool for accurate and timely data extraction.
ChatPDF
ChatPDF, which accepts PDF uploads, was another option I tested. Unfortunately, while it was able to parse the PDF files, it failed to correctly calculate the free cash flow. The errors in its calculations were significant enough to make me wary of relying on it for critical financial analysis. It even failed upon being provided with more detailed instruction. It was clear that while it could extract data from PDF documents, its analysis was not as robust as ChatGPT’s, which was more consistent and precise.
AskYourPDF
AskYourPDF’s free version had a significant limitation: it only accepted PDF uploads of up to 100 pages. While this might be fine for smaller filings, Academy Sports' filings were often longer than this threshold, making it unusable for this particular task. Even when I managed to upload shorter documents, the tool didn’t perform as well as ChatGPT in extracting and calculating the relevant financial data. The limitations in file size and accuracy left me frustrated, and I quickly ruled it out as a viable option for serious financial analysis.
Conclusion: Why ChatGPT Stands Out
After testing these five AI tools, I can confidently say that ChatGPT is far superior for extracting and analyzing SEC filing data, even when using the free version. Its ability to quickly identify key financial figures, such as operating cash flow and capital expenditures, and accurately calculate free cash flow, makes it an indispensable tool for my research. Furthermore, its conversational interface allows for a more interactive and dynamic research process, which is invaluable when navigating complex financial documents.
While other tools like Google Gemini Advanced showed promise in web research and qualitative analysis, none matched ChatGPT’s performance in parsing SEC filings and performing the detailed calculations necessary for assessing a company’s financial health. As I continue to rely on AI tools to streamline my investment research, ChatGPT will remain my top choice for analyzing SEC filings—though, as always, I will adhere to the principle of "AI but Verify."
For investors looking to enhance their research process, ChatGPT offers a powerful and reliable solution. It’s a tool that not only saves time but also helps ensure that critical financial insights are accurately extracted and analyzed. Whether you’re calculating free cash flow, assessing financial risk, or reviewing earnings call commentary, ChatGPT provides a robust foundation for informed investment decisions.
My next article will likely be the write-up for ASO itself - with the assistance of ChatGPT.
Thanks for being part of the Safe Harbor community! Follow me for more insights: LinkedIn | X (formerly Twitter)
Disclosure: This information is provided for informational purposes only and should not be considered a solicitation or recommendation to buy or sell any securities. The author or entity providing this information may hold positions in the securities discussed. This is not investment advice.
Kris, These are invaluable insights you have here. I can only imagine that the AIs will be able to do more indebth research as time goes on, espcially with the advent of AI agents once they get the bugs out.
Makes sense especially for the qualitative aspects. I think there's more progress required on the quantitative side.