What is a Histogram? Why is it Important in Real Estate Data Analysis for Portland, Oregon?

Appraisers endeavor to take the measure of a neighborhood with a variety of charts and graphs. It helps them to frame the value opinion and see how the subject property relates to the neighborhood or market area as a whole. 

A histogram is a graphical display of data using bars of varying heights. In a histogram, each bar groups data for a single variable into ranges (or bins). Taller bars show that more data falls in that range. A histogram may be used to display the shape and spread of the data of the real estate market and illustrate trends of properties that have sold and/or are listed. They’re a great visual tool!

Example

Lot Size

The above histogram depicts the various lot sizes for detached single-family residences in the Ladd’s Addition historic district. The data represents sales and/or listings in the RMLS database from the year 2000 to July 2019. What can we learn from the above chart?

1) There is a tight uniformity of lot sizes in this district. 57% of the data falls into just one bin! 
2) 87% of the data falls into the bins representing lots from 0.08 acres to 0.159 acres (~3,485 sq. ft. – ~6,926 sq. ft.). This shows that large lots in Ladd’s addition are rare.
3) Properties outside the aforementioned bins would require more careful analysis by the appraiser (or agent).
4) Dealing with extreme outliers (properties on tiny lots or larger ones for the district) may require the use of a number of data science and appraisal techniques as comparable properties would be hard to find.
5) There is little developer potential for subdividing lots as virtually no lot is large enough to be reasonably partitioned.

Portland Real Estate Market

As you can see, you can squeeze a lot of information out of just one histogram. These charts can be used to display many different variables: age, gross living area, bedroom & bathroom count, sales prices, etc. By evaluating histograms of several key metrics, one can quickly see how a particular property fits into a neighborhood (or district) as a whole.

The Portland market is eclectic and appraising unusual properties requires the careful application of local expertise, the tools of statistics and data science, and sound judgement.  

Appraisal Reports

If you are a homeowner and are looking to sell your home, you would greatly benefit from a prelisting appraisal. Our firm will bring high-level analytics to your report and give you a sound understanding of the current market.

If you are an agent and need detailed neighborhood analysis, or analysis of specific areas or specific segments of the market, please contact us and we can generate a custom report to help you frame a listing price for your client!

Ladd’s Addition—Neighborhood Analysis of Single-Family Residential Homes (July 2019)

Ladd’s Addition is a unique historic district in the city of Portland that is known for its diagonal street pattern radiating from a central park spoke—the Ladd Circle Park & Rose Gardens. Ladd’s addition is named after the 19th century Portland mayor, William S. Ladd. This historic district is part of the Hosford-Abernethy neighborhood. The street layout is easy to spot from aerial maps of the city and clearly delineates the district’s boundaries.

2018-11-07_18-10-38

Appraisers endeavor to take the measure of a neighborhood with a variety of charts and graphs. It helps them to frame the value opinion and see how the subject property relates to the neighborhood as a whole. For this blog post we will use the venerable histogram to better understand the single-family residential market for Ladd’s Addition.

A histogram is a graphical display of data using bars of different heights. In a histogram, each bar groups numbers into ranges. Taller bars show that more data falls in that range. A histogram may be used to display the shape and spread of the data of the real estate market and illustrate trends of properties that have sold and/or are listed. They’re a great visual tool!

The following information is based on detached single-family residential homes that were sold and/or listed on the open market as reported by RMLS—the primary MLS service for the City of Portland. Data was pulled from the year 2000 to the present date.

Age

Age

Most properties within Ladd’s Addition were built around 1900-1930. This is reflected in the age histogram showing most of the homes falling in the 91-97 years bracket. New/newer constructions in Ladd’s addition is nearly unheard of.

35 different homes that sold over the last 20 years were on the National Register of Historic Places.

Bathroom Count

Bathroom

Most homes in the neighborhood have two bathrooms; but a home with only one bathroom is not uncommon. One home had the equivalent of 5 & 1/2 bathrooms. There is no waiting in that house!

Bedroom Count

Bedroom

Two to four bedrooms is the norm, with a few outliers having either one or 6-7 bedrooms.

Garage Stall Count

GAR

A one-car garage is the norm for Ladd’s Addition. However, a substantial number of homes in the dataset have either no garage at all or an extra garage stall.

Total Square Footage

Total SF

This metric includes above-grade living area and basement space for a combined figure. The median total square footage for the neighborhood is 2,500 sq. ft. There are some larger homes in the area, with 32 sales/listings being at or above 4,800 sq. ft.

Gross Living Area

GLA

Gross living area only consider non-basement living space. Often the market reacts more strongly to gross living area, usually applying a discount to basement space. Median gross living area is approximately 1,700 sq. ft.

Level Count

# of Levels

The majority of properties are three levels in this neighborhood.

Lot Size

Lot Size

The vast majority of properties are on lots 0.120 ac – 0.129 acres (~5,227 sq. ft. – ~5,619 sq. ft.). There is substantial uniformity of lot sizes in this neighborhood.

Sales Price

Sales Price

This histogram includes sales prices over a 20-year period. The highest sales price obtained in the neighborhood (on the open market) was $1,105,000 on 3/17/2017.

# of Sales

# of Sales

Ladd’s Addition doesn’t have a substantial turnover of homes each year. The fewest sales in a year was back in 2015, when only 10 single-family residential homes sold. 2016-2018 averaged 20 sales per year. This year looks to be trending lower. Year-to-date sales are only 8 and, if the trend holds, 2019 may signal a cooling off of the number of sales.

So, there you have it, a brief histogram overview of one of Portland’s most iconic neighborhoods!

Appraisal Reports

If you are a homeowner and are looking to sell your home, you would greatly benefit from a prelisting appraisal. Our firm will bring high-level analytics to your report and give you a sound understanding of the current market.

If you are an agent and need detailed neighborhood analysis, or analysis of specific areas or specific segments of the market, please contact us and we can generate a custom report to help you frame a listing price for your client!

Getting Appraisers to Reconsider Value—Do’s & Don’ts

You’re a real estate agent and after a lot of back-and-forth, give-and-take, offers and counteroffers, you’ve helped your client negotiate a sweet price for their home. All the work getting the property ready for listing, the extensive marketing, the numerous showings, the sometimes tedious offer evaluations has paid off. There has been a meeting of the minds, the contract has been signed and everyone is anticipating a smooth closing. Shortly before the home is set to close a bright light appears in the sky. Streaking through the atmosphere is a meteor scientists have dubbed “the appraisal”; it came of out nowhere and impacts your deal at hypersonic velocity, obliterating it in an instance. You, the seller, the buyer, the lender—heck, even the mailman, are all shocked the deal has fallen apart. What just happened?

If you are a real estate agent, has this ever happened to you? If so, you’ve probably run through the five stages of grief:

1) Denial—there’s no way that idiot appraiser killed my deal! There must be some mistake.
2) Anger—that idiot appraiser killed my deal!
3) Bargaining—I can fix this!
4) Depression—that idiot appraiser killed my deal. It’s not fair.
5) Acceptance—that idiot appraiser killed my deal. We’ll have to cut price or find another buyer.

The purpose of this blog post is to lend some insight into what an agent should do and not do during the third stage. It is quite possible that you can fix this—but only if the facts are on your side. This post is not a list of tricks to pull over on the appraiser in the hopes you can cajole him/her to change their mind. Given that appraising is part art and part science and is an opinion, it may be that there are important considerations the appraiser has overlooked. How best to frame such considerations? How best to convey the facts? That is what we will consider.

It’s important to note that we’re in this situation in the first place because we didn’t get a cash buyer. There is no law forbidding someone from paying more than the average market participant for a home unless the motive for doing so falls under money laundering, bribery, or some other equally nefarious scheme. That’s not the case here. The buyer loves the home and both buyer and seller feel the price is fair. However, anything other than cash means a lender gets involved and a lender means strings come attached. While lenders can differ considerably in underwriting standards and risk tolerance, most will require an appraisal to make sure the collateral supports the amount being loaned. In many cases even if the appraisal is just few thousand short, the underwriter will not go through with the deal or require the seller to cut the price.

If an agent feels the price is reasonable they have the option of requesting the buyer’s lender get the appraiser to rethink their conclusions. Usually this is done via a document known as a “reconsideration of value” (ROV for short).

Don’t Take it Personally or Get Personal—Don’t Go There

Before we get into the weeds defining ROVs and how to craft them, I think it good to say something that should already be universally understood: don’t take the appraiser’s opinion personally and, please, don’t get personal.

While relatively rare, a number of appraisers can attest, including yours truly, that some agents do seem to take the appraisers opinion as a personal affront and get personal in return. I’ve had some agents get abusive with me and send me either a nastygram or leave me a charming voicemail. Don’t send an appraiser a dead fish and don’t question their parentage or general intelligence. Trust me, appraisers don’t want to “kill your deal.” If the appraiser really has messed up, that’s where the ROV comes in. So, let’s define it.

Recap #0:
Do: Remain professional.

Don’t: Get personal.

What is a Reconsideration of Value?

In short, a reconsideration of value is another appraisal. This is an important point. Anytime an appraiser is asked to proffer an opinion of value (even if it is just to reconsider one already made) it is an appraisal. Therefore, an ROV is a big deal and most appraisers take it seriously.

Richard Hagar, a nationally recognized appraiser and valuation expert, points out the various reasons why an ROV may be necessary:

  • To correct a serious mistake of material deficiency in the original report.
  • A means of passing along important information not previously disclosed to the appraiser.
  • A means of getting the appraiser to consider information that was not available during the original appraisal.

If any of the above points are all valid for your deal it could result in a different value conclusion. However, in 9 cases out of 10, an ROV is often used as a vehicle to influence the appraiser’s opinion of value. (Which is a legal no-no.)

Most appraisers are hard-working professionals who spend a lot of time researching and analyzing market data. Appraisal reports can easily balloon to 50 pages or more with commentary and exhibits. Oftentimes appraisers will embed a short essay/commentary about why their opinion of value doesn’t align with the negotiated sales contract. Read it.

All appraisers have stories about questions they get about their reports that are already answered in the narrative commentary. It’s annoying to say the least. Even more annoying is being given a list of possible comparables only to find you either already used those sales in the original report or talked about why you didn’t. Appraisers, while required to remain neutral and objective, are still human beings. An ROV is going to put an appraiser on the defensive right out of the gate since they are being paid for their opinion and now are being implicitly told that their opinion stinks. Most appraisers are loathe to change their conclusions and will be even less inclined to rethink their conclusions if an ROV is riddled with questions or items already dealt with in the report. It’s okay to question an appraiser’s conclusions/findings, but make sure the ROV shows that you know the appraiser already addressed any item you’re questioning if they did, in fact, address it. It shows professionalism on your part and the appraiser immediately shifts their perspective. Oh, I got a serious one here. Let me dig into this.

Recap #1:
Do: Take ROVs seriously.

Don’t: Use an ROV as a frivolous means to see if you can squeeze a little more money out of the appraisal. Read the report!

Speaking the Same Language—Appraisers are from Mars, Agents Are from Venus

Most loans will fall under federal guidelines and use a definition for market value found on the form report workhorse of the appraisal world—the Fannie Mae 1004 Form.

Virtually all agents are familiar with the Fannie Mae form report. However, many agents (and sadly, more than a few appraisers) have not taken the time to read the definitions and certifications contained in the form. It could save everyone a lot of grief. Remember, the lender is the appraiser’s client and intended user. Not even the borrower is the appraiser’s client. The lender has hired the appraiser to produce a report that conforms to the guidelines set forth by Fannie Mae—and that includes using their definition of market value. It is:

DEFINITION OF MARKET VALUE: The most probable price which a property should bring in a competitive and open market under all conditions requisite to a fair sale, the buyer and seller, each acting prudently, knowledgeably and assuming the price is not affected by undue stimulus. Implicit in this definition is the consummation of a sale as of a specified date and the passing of title from seller to buyer under conditions whereby:

(1) buyer and seller are typically motivated;
(2) both parties are well informed or well advised, and each acting in what he or she considers his or her own best interest;
(3) a reasonable time is allowed for exposure in the open market;
(4) payment is made in terms of cash in U. S. dollars or in terms of financial arrangements comparable thereto; and
(5) the price represents the normal consideration for the property sold unaffected by special or creative financing or sales concessions* granted by anyone associated with the sale.

There is a lot to unpack here, but let’s just focus on the idea that market value is the most probable price.

A little time may have passed since your last high school/college statistics course. (My growing number of gray hairs can attest to that for me.) The most probable price doesn’t mean the highest price (or the lowest price). It’s the “Goldilocks Price,” the price that is just right.

If an appraiser uses comparables that adjust out to something like the illustration below, what conclusion would you draw regarding the most likely price the typical buyer would pay for the subject property?

1

If your subject’s contract price is more in line with comparable #4, it would be awfully tempting to fixate on that. Appraisers have a saying: one comp does not a market make. It’s why Fannie Mae mandates at least 3 closed sales be used in a report. They want assurances there’s more than one buyer for the subject at the negotiated price. In the above case, the appraiser’s value conclusion ($350,000) is tightly aligned with 5 out of the 6 comparables used. Comparable #4 may represent the maximum value of the subject, but it is not the most likely price for homes similar to the subject.

You may be 100% correct that the subject’s higher contract price has some market support; but if careful analysis shows that price to be outside the most probable range, the appraiser is contractually obligated to go with the most probable.

But what if the market is rapidly changing? That’s a fair point. In that case, review the report to see if a careful discussion has been made of any applicable pending sales. In my own practice, I try and reach out to agents for pendings (and even actives). I have no problem giving weight to a pending sale—especially if I reached the agent and they confirmed the final price is pretty close to the price advertised in MLS. That’s why its good agent practice to return an appraiser’s phone call or reply to any email inquiries.

(Note: Agents differ in interpreting confidentiality requirements for pending sales. I personally never press an agent. Again, many will at least hint if the final price is above or below the published MLS price.)

Keep in mind: an active listing is like a person looking for a date; a pending sale is like a person engaged; a closed sale is like a person who’s walked down the aisle. An engagement ring ain’t nothing, but it ain’t any guarantee either.

The most persuasive package is a strong mix of recent closed sales, some verified pending sales, and perhaps (if you have it) some signed backup offers on the subject. All of that together should strongly indicate what the most probable price for the subject is. In an ROV, if the appraiser failed to do their homework verifying what the pending sales are doing (or doesn’t even talk about them), you can talk to fellow agents and see if they can confirm some details. Then you’ll have factual and relevant information to include in the ROV. Cold hard facts are your best friends.

Recap #2:
Do: Use a “most probable price” definition of market value.

Don’t: Use a “maximum price” definition of market value.

Out of This World Comps—Well, Out of This Neighborhood

Appraisers are given guidelines from Fannie Mae and then usually given additional guidelines from the lender (or the lender’s agent). Some standard comparable guidelines include:

1) Two sales within the last 90 days.
2) No sales over a year.
3) Comps within a mile for urban/suburban properties; within 5 miles for rural properties.
4) “Bracketing” of most major features. (No across-the-board adjustments.)
5) Adjustments be within a certain percentage. (Fannie Mae actually doesn’t require this and lenders shouldn’t either, but it still lingers in some published guidelines.)

It goes on. (Some lender guidelines are over 10 pages long.)

Rules of thumb are okay and may work in many cases, but sometimes appraisers can get tunnel vision and apply guidelines too strictly and miss the forest for the trees.

I think we can all agree that Ladd’s Addition is a visually striking neighborhood when viewed from a map:

2018-11-07_18-10-38

Imagine you’re the agent for a home in this neighborhood and imagine a report that meets all Fannie Mae and lender guidelines but fails to include a sale from within Ladd’s Addition! If it cuts the price, you wouldn’t be a happy camper. While no appraiser would commit an error that egregious (I hope), all agents are aware of smaller “pocket” neighborhoods that command a premium but may not have had a recent sale.

I often pull all data RMLS has for homes on a subject’s street and immediate pocket area. Yup, all 20 years. This allows me to sniff out if there is some strong locational influence. As an illustration, the graph below shows similar properties from the subject’s pocket area and similar properties from a competing one:

2019-04-14_9-06-13

There have been no recent sales in the subject’s pocket area, but the yellow-orange dots depicting sales from the subject’s area are all above the gray dots from a nearby competing area (perhaps where the appraiser pulled all of their comps). This shows that the subject’s area commands a noticeable premium. Did the appraiser include that premium in the report? If not, again, cold hard facts are your best friends.

However, and a word of caution, appraisers often encounter this problem in reverse. They are forwarded sales from areas that command a premium the subject’s neighborhood or pocket area lacks. (I’ve been sent homes from a golf course community for a property in a standard residential tract area.)

Obvious tricks like that shreds your credibility in the ROV and will cause both the appraiser and the lender to take your request less seriously.

Recap #3: 
Do: Use clear market data that shows the subject’s neighborhood or pocket area commands a premium.

Don’t: Suggest homes from competing neighborhoods or market areas that you know command a premium the subject’s area lacks.

Superior Comparables—Arnold Schwarzenegger vs. Napoleon Dynamite

If the subject property is one of the nicer homes in the market area, the appraiser should take great care to find sales as similar as possible—even if they violate some of the typical lender guidelines. That might mean going further back in time or a bit farther in distance to find the right comparables. I routinely grid sales 12-24 months old and then use time indexing to bring the value current. For acreage or very unusual properties, 36+ months might be needed. Time indexing is an extremely important tool and one all appraisers should be familiar with.

Don’t be afraid to stay in the neighborhood and pull all available data to see how homes like the subject perform relative to the market area. It might be that a clear case can be made for a higher valuation of the subject.

If the subject has sold multiple times on the open market, a review of those prior sales may show that whenever it is on the market, it commands a premium due to its overall quality or some special amenity or feature(s).

However, if a home is deemed an over-improvement for a neighborhood, it likely suffers from an obsolescence stemming from a lack of conformity to the neighborhood. If most homes are under 3,500 sq. ft. and the subject is 6,500 sq. ft. and all similar homes are located far away, then the property won’t likely bring in the same price as those homes in a more conforming area.

A word of caution: appraisers are used to getting vastly superior properties suggested to them in the hopes of making the contract price. You may have seen this illustration floating around some of the agent forums:

2018-12-15_18-40-48

Don’t do that. If your home is Napoleon Dynamite don’t suggest Arnold Schwarzenegger. Again, it reduces your credibility in the eyes of the appraiser and the lender. We need to compare apples to apples as much as possible.

Recap #4:
Do: Suggest similar properties—even if a bit dated or a bit farther away than typical.

Don’t: Suggest vastly superior properties that are in a different market segment than the subject.

Fuzzy Numbers—Did They Remember to Carry the 1?

Appraisers typically use some type of dedicated software package to “grid” comparables and make adjustments for superior or inferior features. While there are some error correction tools, they don’t always catch all mistakes. An examination of the sales grid might show that adjustments are not mathematically consistent. Check the numbers.

Broader issues with the numbers in the grid might be how the adjustments were derived in the first place. That’s a complex topic (and one for another blog post). But check comments in the report to see how some of the more important adjustments were derived.

A word of caution: don’t nitpick. If the appraiser didn’t adjust for a small shed, will that really make or break your deal?

Recap #5:
Do: Check adjustments for mathematical and logical consistency.

Don’t: Nitpick or quibble about minor amenities or features.

What Can You Do Proactively?

If you have an atypical property on your hands, you and your client would greatly benefit from a prelisting appraisal. Having an appraiser give a value opinion prior to listing can help with setting the right price and can point out issues ahead of time. You’ll get a list of comparables an appraiser feels is most relevant to the subject. (You’ll also get an accurate sketch of the subject’s size and dimensions.)

Some agents fear getting a prelisting appraisal “locks” them in somehow. It doesn’t. You may keep it in your back pocket if you want. All appraisals are confidential.

*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~

I hope this article is somewhat helpful for agents. There are many other aspects to discuss about ROVs and I will write about the topic some more in the future.

Click here for a one-page summary of this article.

An Appraiser’s First Impressions of Clear Capital’s AVM

Collage_HD 2018-12-27 23_53_58

It’s no secret the lending industry is aggressively pursuing alternatives to traditional appraisals. While most participants in the industry will concede an appraisal is still the gold standard for collateral valuation, they increasingly see the appraisal as something to be steered around or avoided altogether. An array of products/alternatives are being bandied in lieu of a traditional appraisal: Hybrids, Evaluations, BPOs (broker price opinions), Desktop Appraisals, and outright Appraisal Waivers. The industry is also turning its attention more and more to AVMs (automated valuation models).

AVMs have been around for nearly three decades now. The first AVMs had a hard time with consistency and accuracy, but as the world has become increasingly digital and data driven, the accuracy of the models has been steadily increasing. In conforming tract home subdivisions with sufficient recent sales, their value may be extremely accurate.

An appraiser should keep an eye on alternatives to the services he/she offers, so I recently downloaded a number of white papers on AVMs from different websites, including Clear Capital’s. I was somewhat surprised that Clear Capital’s white paper freely admitted that an AVM is not as good as an appraisal, but could be a starting point in risk assessment and collateral valuation. I had to provide my email address and phone number to access the white paper. This resulted in me receiving a follow-up call from Clear Capital. (Full disclosure: I previously worked for Clear Capital for a couple of years doing quality assurance work on incoming appraisals.) The gentleman on the phone was very nice. We chatted a bit and I asked him how long the product has been offered and he mentioned they took their previous AVM off the market and just released a new and improved model a couple of months ago. I asked him if they needed beta testers and he said no, but he did email me a code to sample five AVM valuations for free on their website.

Clear Capital AVM Sample Redacted

That day I had just completed an appraisal report and, after completing it, ran the subject property’s address through their portal. The AVM is highly specific in its dollar output: $416,771, which I thought was a bit weird. It might be their way of emphasizing this is an algorithmic valuation. If so, that is the opposite of what we as appraisers do, which is round an opinion of value to emphasize a measure of uncertainty. (I sure hope no licensed appraiser is signing their name to an opinion of value as specific as $416,771!) For reference, my developed opinion of value was: $400,000, so the AVM was ~4.2% higher.

The AVM output is very bare bones: just three pages and mainly a list of addresses. I got 30 addresses in total; 15 apparent sales and 15 apparent listings; I say apparent because, oddly enough, there is no legend anywhere in the document explaining what the colors denote. The AVM gives a high estimated value of $451,500; a low estimate of $390,000; and the official, highly specific, estimate mentioned previously. It also provides a confidence score with a letter next to it: “H,” “M,” or “L” to presumably denote either high, medium, or low confidence. (My AVM report had an “H” next to the confidence score.) A map of the properties is provided on the second page. The third page consists of just two paragraphs of disclaimers: first warning the product is not an appraisal and then warning the reader not to reverse engineer their work or infringe on their intellectual property. With so little explanatory material in the document itself, I don’t think they need to worry about anyone being able to reverse engineer their system.

All the sales listed are within approximately one third of a mile of the subject property and the dates of sale within ten months of the date of the AVM report. (The listings are selected from within three quarters of a mile.) Five of the seven comparables I used in my report are found on the list, but none of the comparables I used cracked the top ten on either the sales or listings tally. My subject is a single-level home and all the properties I gridded in the report are single-level as well. Of the top ten sales the Clear Capital AVM used: five are two-story structures; one was not listed in the local MLS; another was listed but had no photos and no agent comments at all; one was a bit older and smaller; and the other two would have made acceptable alternative comparables in my report.

As I mentioned before, there is almost no explanatory material with the AVM output, so I really don’t have a clue how it derived its value estimate. I checked to make sure the AVM didn’t do something as simple as average the 15 sales (or the 15 sales with the 15 listings). It didn’t.

So, is something like this useful? I think so. While many of the properties it listed were rejected by me in my comparable search, the results were within 4% of my opinion of value. As their own white paper stresses—and the disclaimers in the AVM report itself reiterates—the AVM output is not an appraisal. The cost of the AVM would have been $10 if I didn’t have the promo code. I can see lenders using products like this when the loan-to-value ratio is low and the credit score of the applicant is high. (A full-blown appraisal may be overkill in that situation.) A $10 charge is also nominal considering the total fees a mortgage or other real estate loan product may generate. A quick peak at the AVM early on may give a loan officer an idea of what they’re in for or if the deal is feasible or not. (It is hoped said loan officer is also paying mighty close attention to the confidence score.)

(I am glossing over the bigger question as to how these products will be checked for accuracy, data integrity, tamper resistance, etc. But that is a blog entry for another day.)

While my opinion may be considered biased for obvious reasons, I genuinely believe appraisers offer an extremely valuable service to the public and are a pillar in sound collateral valuation and risk mitigation. However, it is clear that the future of valuation will include increased utilization of AVM products such as the one Clear Capital offers or the free Zillow “Zestimate.” Appraisers who work in areas with highly conforming properties will need to make sure they level-up their skill set to specialize in the valuation of complex properties; the type that currently gives computer algorithms nervous breakdowns. I strongly suspect the appraiser of the future will be, essentially, a data scientist. One that has strong Excel, R, Python, and/or SAS proficiency. There may be fewer of us in the future, but the ones that remain will be extremely talented professionals who offer local, boots-on the-ground expertise with high-level analytical skills. And we can offer what no computer currently can: a friendly smile and a listening ear as the homeowner tells the story of their property!

Using the R Programming Language to Produce Correlation Matrices & Correlograms for Residential Appraisal Reports

As I mention in the bio for this blog, one of the most influential individuals in my professional development as an appraiser is George Dell. He is a nationally recognized valuation expert who teaches a method he calls “Evidence Based Valuation©.”

(If you’ve never taken a class from George, stop reading this blog post, go to his website, register for one of his classes—oh, it’s on the other side of the country? So what? Register!—and then resume reading this post. You’re back? Great.)

George’s classes emphasizes reproducible appraisal findings and deemphasizes subjectivity and guessing. Reports that follow George’s tenets are clearer, more logical, and more convincing to the end user. I can personally attest to a substantial increase in work quality.

George is on a bit of a crusade to get appraisers to use more sophisticated analytical tools. One such tool is the programming language R.

R can be a bit intimidating for appraisers—even for those who possess advanced Excel skills. However, one does not have to be a seasoned computer programmer to immediately start using it to produce charts and graphs that can aid in analysis and improve a report’s quality.

Let me walk you through the steps needed to produce this chart, known as a correlogram:

Rplot

 

Per the STHDA website, a website that shares information on statistical tools, a correlation test is “used to evaluate the association between two or more variables.” Typically, a number is assigned that can range from -1 to +1. If two variables, say “Close_Price” and “Total_SF” have a correlation of +0.74, that means there is a strong positive association between the two. (In reports, this is often expressed as a percentage.) If the number were negative, it would mean as one variable goes up the other goes in the opposite direction. A good example of this in my market is the relationship between “Year_Built” and “Acres.” Newer homes tend to be on smaller lots as often a larger lot is purchased and partitioned by a developer.

A correlation matrix, per STHDA, “is used to investigate the dependence between multiple variables at the same time. The result is a table containing the correlation coefficients between each variable and the others.”

A correlogram is a visualization of the correlation statistics.

Once you get the hang of the process, you can start producing correlograms in under 5 minutes using R. It allows for fantastic support in an appraisal report.

Correlation matrices do, however, have their limitations and need to be used carefully. Again, I strongly recommend taking George Dell’s “Stats, Graphs & Data Science¹” class to obtain a better understanding.

Without further ado, let’s begin:

Step 1:

Install the R programming language: https://cran.r-project.org/

R runs on Windows, Mac, and Linux.

Step 2:

Download R Studio (a graphical overlay to R) from this site.  R Studio is a free program. Make sure you choose the desktop option. (If the site is asking you to pay $30,000 a year, you accidentally clicked on the commercial license product . 😊)

Step 3:

Install the “PerformanceAnalytics” package.

Click the “Packages” tab in the bottom-right pane. Click “Install.” Type in the name of the package (it will autocomplete based on what is available on CRAN).

2

Note: You can customize the look of R Studio, I’ve chosen a darker theme as it is easier on the eyes, so don’t be concerned if my screenshots don’t exactly match what you see.

Step 4:

Open the R script file that contains the code.

You have a number of options here. To save yourself time, you can simply click this link to download a prepared R script file I am hosting on Dropbox. On Dropbox, click the download button on the upper right-hand corner and you will get a file that looks something like this, depending on your folder view settings:

2018-12-07_13-30-07

When you double-click on the R script file R Studio will automatically open if it is closed or, if already open, simply add the code to the top-left pane. The code is just three lines long and will look something like this on your display:

zzzzz

The first 5 lines in blue are just comments I’ve added and are not code. By putting a “#” sign in front of comments, I am telling the program not to treat the text as code.

If you prefer not to download the prepared file, simply copy the text from the Dropbox display and paste it into a new R script.

Step 5:

Import the data from an Excel file. Click on the “Import Dataset” tab in the top right-hand pane, and click “From Excel…”:

a

If you try to import a CSV file you’ll get this error:

CSV

Future versions of R Studio may be able to handle CSV files directly, but for now, just make sure the file is an actual Excel file.

You’ll get a preview of your data that looks like this:

b

For appraisal work, you’ll want to start off working with variables that can be put in the “double” type format (a computer variable type that permits greater numerical precision). The type of variables we will want in our correlation matrix are, in brief, ones that are interval or ratio variables like “Close_Price” where it makes sense to say that a home is twice as expensive as another. (Working with a class of variables known as “categorical variables” is beyond the scope of this blog post—a Q3 home is not twice as nice as a Q6 home.)

I’ve made some custom Excel templates that allows me to take the data exported from my local MLS and it put it into the format I want. I then save just that tab’s information to a separate file labeled “Correlation Data.”

Pro Tips: a) For column headers, use the underscore to separate words rather than a blank space. So, “Total_SF” rather than “Total SF” is safer. b) Don’t use the “#” sign for any headers as that can mess up some R package’s ability to interpret. So rather than “#_Acres” put “No_Acres” or just “Acres.” c) Make sure “Close_Date” is in Excel “serial date” format.

Once, you’ve looked the data over, simply click the “Import” button at the bottom right to bring it into R.

Step 5:

Highlight the three lines of code and click the “Run” button:

c

Well, that part was easy!

(If you highlighted the comments above the code, it will have no effect on the outcome.)

Your graph will appear in the “Plots” tab at the bottom-right pane. Simpy click the “Export” option and you can save as an image or PDF and stick it in your report:

d

Once your templates are all set up, you can produce this chart in just minutes.

Correlograms can be very powerful. Reading from the top row, “Close_Price,” and reading across allows you to see how all the other variables correlate with “Close_Price.” If, “Acres” and “Close_Price” show virtually no correlation, you can point to the correlogram as proof that no lot size adjustment is needed.

There are many different types of correlograms that you can do with R. In a future post, I’ll review a number of them!

Let me know your thoughts in the comments section. If you have a question, don’t hesitate to ask!

And, finally, take George Dell’s class!