AVMs – The State of the Art

In astronomy, heliocentrism is the theory that the sun is at the center of our solar system. We can look back now and chuckle over how, for so many centuries, heliocentrism had such a tough time gaining acceptance. Apart from religious-based resistance, some would argue against this view with such common sense as “if the earth were spinning and moving around the sun, people and objects would tend to fall off.”

Perhaps some day, years from now, we will share a similar chuckle over the way our industry had a tough time integrating AVMs into the origination process.

In the never– ending quest to wring cost and time savings out of the mortgage origination process, lenders are looking to Automated Valuation Models (AVMs) as an alternative to appraisals to establish property value. The allure of AVMs lie in their clear advantages over the traditional appraisal process – they are cheaper, faster, and more objective than appraisals. AVMs are not without a downside, however. They present a potential for extreme error, a fear of the unknown, a lack of widespread acceptance, and conflicts with established processes. Solving those issues will require testing, standards of use, product development and refinement, and process evolution, all of which are occurring daily in the mortgage industry.

Benefits of AVMs

Few people deny the attractiveness of AVMs. The lower cost is certainly significant. Typical retail pricing for an automated valuation runs from $25 down to below $10. For high volume users, the pricing runs even lower. Compared to a typical price of around $350 for a full appraisal, the potential cost savings from using AVMs is dramatic. The time savings can be equally compelling – almost instantaneous feedback versus waiting a week or, particularly in high volume periods, several weeks. Finally, while not so easy to measure as time and money, the objectivity of AVMs is a compelling feature for many potential users. The lack of human intervention makes them especially useful for combating fraud.

Drawbacks

Given those dazzling advantages, it is no surprise that people have been quick to forecast the demise of the traditional appraisal. However, AVMs have made little progress in supplanting appraisals for the majority of first– lien lending decisions. AVMs, it turns out, have some drawbacks, too. One drawback is the potential for significant error. While appraisers can certainly overestimate the value of a property by a large amount, more often than not that is the result of fraud – and it is a rare occurrence. An automated model, on the other hand, may make extreme overvaluations regularly, depending on the data provided to it. With appraisal fraud, a lender can simply say, “Well, I’ll never use that vendor again.” It’s not so easy with AVMs – every vendor available will produce some bad values on a small yet consistent basis.

Another downside to an AVM valuation is the inherent inflexibility that can be thought of as “the flip side” of the benefit of objectivity. The mortgage industry has built its processes around the concept of exact property values. The borrower purchases the house for $150,000 and the maximum LTV for the loan program is 90%, so the loan amount can be $135,000. If the appraised value turns out to be less than $150,000, even by $1, the loan no longer qualifies for the same program. The appraisal process allows for a skilled person to effectively make the statement that $150,000 is a supportable value. An AVM process might stubbornly insist that, $149,999 is indeed the right amount. The bottom line for most mortgage originators is that, if the AVM process results in deals falling through because of such inconsequential valuation differences, the savings of time and money are rendered meaningless.

Lastly, much of the reluctance to use AVMs can be attributed to the “unknown” factor. We have decades of experience with appraisals and how loans perform based on appraised values. Sure, appraisals have flaws, but we know what those flaws are and we can predict their impact. With automated models, we face greater uncertainty. That uncertainty translates to risk that, in the absence of experience, is impossible to measure. Consequently, those responsible for measuring the risk – rating agencies, mortgage insurers, securitizers – must proceed cautiously. Until these participants gain confidence in their ability to assess how AVMs will impact mortgage performance, they will be reluctant to broadly accept automated valuations in lieu of appraisals.

Performance Testing

Fortunately, automated models by their nature allow for more rigorous statistical testing than do appraisals. There are several industry participants who have carefully developed their testing protocols to measure the accuracy and utility of AVM values. Acting together in a forum called the Collateral Risk Consortium, these participants published a description of best practices in testing AVMs. Many of these same entities are actively contributing to the Joint Industry Task Force (JITF) on AVMs, whose goal is to promulgate standards for the testing and use of automated valuation technology. Participants include representatives from mortgage lenders, mortgage insurers, the GSEs, rating agencies, property valuation service providers, appraisers, and assessors. One of the primary tasks of the group is to establish a statement of best practices for appropriate evaluation of AVM performance.

While there are some points of disagreement about how best to evaluate AVMs, there are also some aspects on which most people agree. The fundamental task is to select a population of loan applications for which there is an established property value, obtain the AVM values on those same properties, and compare the established values to the AVM values. One key element of the process is to select recent transactions and ensure that they have not found their way through the recording process into the AVM vendor’s database. Obviously, if they already know the answer, AVMs are very accurate! The goal of the test is to learn as much as possible about the distribution of errors of the AVM, as illustrated by Chart 1. Ideally, that distribution should have a high central peak at the established value, with very small tails on either side. The higher the central peak, the more accurate and reliable is the AVM.

Chart 1

As a mortgage insurer, MGIC has a signficant interest in the performance of collateral valuation methods. The Analytical Services group at MGIC has been developing its AVM testing procedures for several years. Chart 1 illustrates the most recent results for several AVM models on a national sample of properties. Looking deeper into the data, we find that AVM performance varies considerably across several dimensions. For example, the availability and quality of publicly available property data varies considerably across counties, causing significant AVM performance differences. Another factor closely correlated with AVM accuracy is the degree of heterogeneity of properties in a given area. Generally speaking, the more heterogeneous the properties in a given neighborhood are, the less accurate the AVM values will be. Properties that are significantly higher or lower than what is typical for an area are notoriously difficult for automated valuation. In Chart 2, we observe the significant increase in error as prices move both below and above the median for the county in which the property is located.

Chart 2

Standards for Use

Beyond testing AVMs for accuracy, there is a need for consensus on the use of AVMs among the various interested parties. Investors, insurers, and regulators are all concerned with standards for how AVMs will be used in the origination process. Of particular concern is the possibility that mortgage originators will “shop the AVMs,” purchasing multiple valuations until they find one that meets a target value. Passing only the highest of several values to investors and insurers could be significantly misleading. The JITF is developing standards for how AVMs should be used in the mortgage origination process and what information should be transmitted to investors, insurers, and regulators. The future success of AVMs as a mainstream component of the mortgage origination process depends on the development and acceptance of these standards.

Future Considerations

Even with the development of rigorous testing and accepted standards of use, AVMs still face the challenge of fitting into the established process of originating mortgages. As can be seen in Chart 1, all of the models, regardless of how tight their error distribution, have approximately a 50% probability of producing an estimated value that is less than the actual value. This means that going into a purchase transaction with an LTV that is right up to a product limit, you can expect an AVM to return a sufficient property value only half the time. Given that there are many instances where an AVM may return no value at all, the actual rate of success will be even lower. One method of dealing with this problem is to allow AVM values that are “close enough,” typically within 10% of the purchase price. However, most underwriting guidelines require the use of the lower of the purchase price or “appraised value” in calculating LTV. Unless underwriting guidelines are changed, the use of AVM values for the origination of first lien mortgages will continue to face this challenge.

Another possible evolutionary path involves additional rethinking of traditional guidelines and processes. Rather than returning a specific dollar value, for example, an AVM could return the probability that the purchase price is less than or equal to the fair market value of the property. Or, on a refinance, the AVM could return the probability that the LTV is greater than 100%. Underwriting guidelines would then be based on this probability instead of the traditional measurement of LTV. Such products are readily available from AVM vendors and would make automated valuations more useful to the origination process, but implementing such a change would be nothing short of a paradigm shift for mortgage underwriting. Given the compelling benefits that AVMs bring to the table, we believe that, over time, this could happen. Despite the many parties that must reach a mutual agreement, the concept of using AVMs in this way may ultimately prove easier to accept than some other great concepts such as, say, heliocentrism.

article written by Ted Durant

MGIC's Real Estate Valuations Department is a reseller of multiple AVM models as well as BPOs, Appraisal Desk Reviews and inspections.