Highlights

• The Equity Analytics team achieved a 20% decrease in monitor deployment and maintenance as well as 50% faster alert-response time for certain models using Monte Carlo. They are also leveraging Monte Carlo’s Agent Observability capabilities to better assess reliability of AI initiatives like earnings summarization and model suitability. 
• The Global Risk team saw a 20% decrease in time spent on monitoring operations, making standard checks much more efficient and freeing up analyst time for higher-value activities.
• The Medalist Rating team accelerated their data validation process by 20%, allowing them to invest more time developing new Medalist Rating enhancements. 

The Data Quality Gold Standard

Morningstar publishes forward-looking ratings to help investors make critical decisions about their investments, portfolio allocations, and overall risk management. 

These rating systems often include quantitative analysis powered by sophisticated models and billions of individual data points. Data quality is non-negotiable.

Operating at this level of precision requires extensive validation. This typically includes steps such as input and output auditing, transformation reconciliation, and process review.

In the last year, these efforts have been dramatically accelerated by Monte Carlo. The data + AI observability platform simplified anomaly detection and validation while also allowing Morningstar to further develop its rigorous process for proactively handling and resolving alerts. 

As a result, the data validation process has become much more efficient while discrepancies are identified earlier in the process. The time saved is being reinvested by the analytics team into building new and increasingly sophisticated outputs like private-asset fund liquidity and fee comparison metrics and validation models such as those supporting the new Medalist Rating enhancements launching in the spring of 2026.

Equity Analytics: Ensuring Trusted AI Summaries

Behind this expansive coverage is Francie O’Neil and the broader Equity Analytics team, who are responsible for ensuring the integrity and reliability of model outputs as well as the data that feeds the models. 

“The Earnings Summarizer runs on AWS Bedrock, giving the team a managed, scalable foundation for their AI workloads—and Monte Carlo’s Agent Observability sits on top to ensure those outputs remain reliable and compliant. This agent turns quarterly earnings into concise summaries that are published alongside financial ratings. Monte Carlo evaluation monitors alert if an agent has potentially gone against its instructions and included language that could be construed as financial advice in that summary, so the team can further review the relevant output.

“We’re excited about gaining additional insight into our agents and further leveraging Monte Carlo’s agent observability capabilities. It should give us better insight into the reliability of our agents and whether we are using the right models,” said Francie.

For other models, the Equity Analytics team needs to quickly validate hundreds of thousands of data inputs. In the past, this involved setting specific thresholds unique to each company, which was toilsome.

“The amount of time we spend looking into potential issues has decreased because, when we can, we use Monte Carlo’s machine learning anomaly detection monitors,” said Francie, a quantitative analyst at Morningstar who leverages Monte Carlo on a daily basis. 

“Having the anomaly detection monitors automatically train on the historic data at the company level is very helpful because one company is going to have more variance than other companies, and now we don’t have to make those tweaks. You spend less time on monitor upkeep.”

While Francie estimates the Equity Analytics team has become about 20% more efficient in deploying and maintaining their data reliability monitors, one of the largest benefits has actually been the reduced time required to resolve these issues.

“The time we’ve spent looking into alerts has become much more efficient because Monte Carlo points you exactly to what table is failing, what specific segment is failing, whereas before that required an analyst to execute several different queries,” said Francie. “The response time has generally been reduced by about 50%.”

Medalist Rating: Accelerating Validations

The Morningstar Medalist Rating is a forward-looking assessment of mutual funds, exchange-traded funds, and other managed investments, including semiliquid funds and 529 college savings plans. Morningstar assigns Medalist Ratings to more than 350,000 share classes globally as of November 2025.

Marcelis Shaw, a quantitative analyst working on model validation at Morningstar, drives the validation efforts for the Medalist algorithm that extends the qualitative ratings of Morningstar’s analysts to identify and rate other funds that share similar traits. 

Previously, each team member would validate different components by manually writing code contained in individual Jupyter notebooks or by leveraging open-source testing libraries.

“It required a lot of manual work and follow up,” said Marcelis. “When an item would flag, there would be many conversations to figure out next steps, which required the person who created the monitor to sound the alarm manually; the burden would be on that specific, single individual.”

Today, Monte Carlo alerts are visible across the team. Those alerts can then be easily escalated to JIRA tickets to kickstart the resolution process.

“It’s been efficient to have one platform where we can see which validations passed, failed, and attach JIRA tickets to failed validations directly in the system,” said Marcelis. “We moved to a more process-oriented team approach, which has allowed us to scale efficiently across 300,000 share classes every month.”

By consolidating validations within Monte Carlo, which provides centralized access and automated alerts, the team has become more organized and structured in its validation approach. The system has enabled a 20% reduction in time spent on output validations for Medalist Rating enhancements going live in April. This process can now be completed within four business days instead of the five it typically took. Looking ahead, this efficiency gain will allow the team to dedicate more time to developing new analytics for other products.

The other benefit? The time to get new team members up to speed on the data validation process has been reduced by months. 

“We have new analysts that rotate to different teams across analytics. They are able to create validations in the first week using Monte Carlo, whereas before it took weeks to pass the knowledge base to them to create validations in Jupyter Notebooks,” said Marcelis.

Global Risk Model: Catching Issues Earlier

Ahmad Bhatti, quantitative analyst at Morningstar plays a key role in ensuring this large and complex model runs smoothly on accurate data. 

There are eight major sequential processing steps with the longest step taking approximately eight hours to complete. If an issue is found, the entire step needs to be rerun.

“With Monte Carlo, we’ve been able to run our checks 1 to 2 days earlier to catch completeness and accuracy issues, which allows us to resolve issues more quickly and avoid publishing delays,” said Ahmad. 

Automating standard checks, previously done manually, has reduced the time required to validate data by approximately 20%.

“The biggest benefit for our team has been monitoring tables that are necessary to look at, but they’re much less prone to error. Previously, someone was looking at each input going into the model each time and now those checks are done by Monte Carlo monitors,” said Ahmad. 

The Next Chapter: Scaling Confidence in Every Rating

For Morningstar’s Medalist Rating team, the next chapter is about continued innovation. With a more automated and proactive validation process in place, the team is reinvesting time into further enhancing and refining the methodology validation process.

Francie and the Equity Analytics team are excited to continue exploring lineage and deploying monitors further upstream to catch issues even earlier, while the Global Risk Model team is exploring how to expand its monitors across both model inputs and outputs.

Across all three groups, the direction is clear: expand coverage, increase sophistication, and accelerate delivery—without ever lowering the bar for data quality. 

##

Our promise: we will show you the product.