Some excerpts from a conversation on Twitter this evening:
A bit later, after more conversation . . .
The issue here is the possibility of RPR participating in the RETS with the aim of using the RETS for the RPR public records API. RPR’s willingness to consider participating in standards development is good news.
This could be the opportunity to roll some simplified web services APIs into RETS, because I’m not sure it makes sense to use RETS 1.x for the RPR public records API unless the public records are not standardized and lots of metadata is needed. Where I think RETS could play a very important role with RPR is by offering participating MLSs a repository for exchanging data with their data sharing partners. I’m learning from my hero Kristen, one step at a time, and I’m very appreciative that RPR is listening.
I attended a presentation from Dale Ross and Marty Frame to a variety of MLS and public records software vendors last Sunday at the NAR Convention in San Diego. Here are a few points I took away from the Q&A session:
There were questions from other vendors as well about the APIs and the business model, but the above were the highlights for me. I walked away from the meeting thinking they were missing a lot of the potential for how a system like RPR could help MLSs and their broker and agent members. Helping MLSs improve data quality, data standards, and data sharing are all key benefits not being addressed by RPR. Hopefully this will change over the coming months as MLSs negotiate licenses with RPR and require that these issues be part of the deal.
——————————————————————–
Related and recent posts RPR by others:
Brian Larson — Report of RPRs Birth Is An Exaggeration
Rob Hahn — No More Drama and Hype: Known Facts on RPR
——————————————————————–
P.S. Can I please request that all NAR conventions from here on out be in San Diego? What a perfect location, with a great convention center, great hotels nearby, awesome restaurants, entertainment, and, of course, the views and weather!
Last Friday, following the burst of rumors that NAR had purchased Cyberhomes to power its RPR (Real Property Resource) and HouseLogic web sites, the cries of the death of the MLS have risen to a fevered pitch again.
Rob Hahn, founder of 7DS Associates, thinks MLSs need to be very concerned with RPR and that a war is coming between NAR and the MLSs. Brian Boero, from 1000Watt, doesn’t use such stark language, but he, too, thinks RPR will be a “a significant shock to a system that needs it” and asks: “Will MLSs play ball?” He then answers with uncertainty: “As with most things in this space, the outlook is unclear.” As an agent and one of the members of the original NAR PAG that envisioned the RPR, Jim Duncan is excited by the possibilities of RPR and HouseLogic and asks some important questions, namely what will agents be able to do with the information and who will have access to it. The comments to all three of the above posts focus in on the impact the RPR will have on MLSs.
Though I was initially going to wait for an official press release from NAR (I hear it’s coming Monday), the more I thought about it, the more it seemed appropriate to weigh in on some of the speculation about RPR’s impact on MLSs. I’ve already written at length about both the death of the MLS and RPR (RPR or Ready?). In fact, most of the posts on the FBS Blog have been about these same issues in one way or another.
For example, I wrote some time ago that MLS is about more than technology. This crucial point — that MLSs enable competitors to cooperate — is where I think Rob Hahn goes awry in his post. He assumes that the MLS is only about the software. What Rob ignores is that the software merely implements a wide variety of business rules that were carefully crafted by the local MLS or Association in order to create the compromise that makes it possible for the competitors to aggregate their data in the first instance. NAR understands this, though, which is why they’ve been saying that RPR is not intended to replace local MLSs and, to my knowledge, has no listing input or maintenance functions so far. In other words, the data is going to come from the local MLSs and not brokers or agents directly.
NAR and its leadership knows how “local” MLSs really are. The local Boards and Associations and their respective MLSs (some independently owned, others not) are run by brokers and agents competing in the local marker and who have cooperated together just enough to make aggregation of their listings possible. Often, this cooperation results in a complex set of business rules.
This set of business rules is the heart of the MLS today and so talking about the death of the MLS means killing these local business rules in favor of business rules established by RPR. NAR knows this is a tough challenge, and so they’re just not going there, yet. As this issue (death of the local MLS business rules) is debated again in the coming years, I believe there are few core questions:
If you’ve read my earlier posts on these topics, you already know that my answers to these questions are:
All three of these issues are related to each other. FBS serves over 100 different MLSs across the country. We know how important our system is to our customers and that it’s important to have lots of cool features. More importantly, however, we believe responding to each of our MLS customers’ needs is what differentiates us over the long term. We provide value by implementing their business rules, which makes the cooperation central to MLS possible. Responding to these local needs is critical.
At the same time, the lack of standards created by local control of MLSs causes pain for many bigger brokers, franchises and others dealing with multiple MLSs. This is where standards could be of great benefit. Creating standards, however, is very, very difficult, especially at a national level. Of course, one way to create a national standard is to create one MLS. The problem with that result, however, is that you create a monopoly. I’m pretty sure what online real estate needs is more competition, not less. Standards enable competition, monopolies do not.
Accordingly, I applaud NAR buying Cyberhomes and using it to power RPR. As an MLS vendor, FBS loves competition and looks forward to learning more about RPR and how their tools can help our customers. We’d love to see RPR help create a universal property ID. We’d love to leverage data from RPR if it can in any way help improve accuracy of the MLS data. We’re excited about the possibilities of RPR but also strongly believe that those possibilities should not be used to create a monopoly that squelches competition in the MLS software space.
For those of you interested in these issues, join me at the NAR meetings next week in San Diego for the Future of MLS panel. The RPR and related issues should be a hot topic.
Here are some tips from Adam Bosworth (who worked on ODBC, XML, etc.) on how to build a successful standard:
1. Keep the standard as simple and stupid as possible.
2. The data being exchanged should be human readable and easy to understand.
3. Standards work best when they are focused.
4. Standards should have precise encodings.
5. Always have real implementations that are actually being used as part of design of any standard.
6. Put in hysteresis for the unexpected.
7. Make the spec itself free, public on the web, and include lots of simple examples on the web site.
To all those involved in trying to create RETS data (not transport) standards, how many of these have we followed?
(h/t Joel Spolsky)
The Real Estate Standards Organization (RESO) announced the results of the Board of Directors election this last week, and I’m very happy to report that Troy Davisson from FBS was elected to a two-year term. Troy has been a tireless advocate of RETS and contributed to many work groups over the years. Troy brings both server and client developer perspectives to the Board and will make a great contribution.
Congratulations also to Mary Frances Adams (Trend MLS), Paula O’Brien (consultant), and Steve Clarke (Marketlinx) on being elected, and a big shout-out to Kristen Carr (Bridge) for being the first RESO director to be re-elected. Here’s the entire RESO Board for 2010:
Mary Frances Adams – New
Pat Bybee
Kristen Carr – Reelected
Steve Clarke – New
Troy Davisson – New
Sergio DelRio
John Holley
Mark Lesswing
Chris McKeever
Paula O’Brien – New
Paul Stusiak
Mark Wise
I’m hopeful this Board is able to leverage the energy in the community around data standards and prioritize this work to provide a path for MLSs wanting to standardize. I’ve mentioned this many times, but there are efforts going on in many parts of the country and all these efforts would greatly benefit from some standards to start work from and point toward.
Here’s an email I received today from a member of the Board of one of our MLS customers:
Hi Michael,
I believe you are involved with the Data Standards group at NAR, right? We are still struggling with some definitions and wondering if there is anything published by the data standards group on things that have already been agreed on at the NAR level. Do you have anything like that? Yesterday we struggled with “Single Family Detached” home for over an hour. Is there some help out there?
Thanks, and look forward to seeing you in San Diego.
I typically discuss data standards in relation to MLSs wanting to share data or regionalize, but this is a good example of an MLS simply wanting an easier way to get the data right. I pointed this customer to the RETS schema, but also had to say that it wasn’t user-friendly and didn’t say anything very specific about her question.
From my perspective, a broker asking for a definitive data definition to be published at the national level isn’t too much to ask. I would really like to be able to respond, Yes!, here’s a link to the data dictionary we’ve been working on for the last many years and the definition you’re looking for is on row 28 (or whatever). Wouldn’t that be great? Let’s make it happen.
In my last post, I mentioned that RETS data standards may get a boost from the COVE Group of MLSs starting to work together on further defining standard field names for their MLSs. Standard names have existed for a long time in RETS but they are rarely well populated. This new effort is to reinvigorate standard names and try to figure out how they can be changed so they are actually put into use.
Because there are other regional data standardization efforts across the country, I suggested in my last post that sharing the process for this work with the broader community would be helpful so that the work can be coordinated with work other groups are doing. One of the most basic first steps is defining a format for publishing the agreed upon fields. Spreadsheets tend to be the easiest way for the broadest group of people to access the results, and so they are most commonly used in data standardization efforts.
One of the challenges with spreadsheets, though, is that they are two dimensional and the listing data often is nested, especially when it comes to fields with a list of values. How do you best show the field name and the list of values? One approach is to combine them into one sheet and another is to include the list of values on a separate sheet as shown in the example below (though I couldn’t figure out how to link the two sheets together in Google documents).
Another question is whether to include the data type in this initial effort. In the past, defining data type has created a lot of consternation where, for example, some MLSs use a list of values for number of bedrooms while most others use a numeric value. If this effort is to have meaning and usefulness long term, I think we have to define data types as well as names. Lastly, even though RETS schema doesn’t define fields by property type, I suggest the spreadsheet contain a property type column to define those fields particular to certain property types.
I think using Google Documents or some other web based spreadsheet would make sharing and publishing the information on sites like RETS.org much easier for everyone. Sharing is critical for getting input and feedback, and building consensus around the results. For example, if you think of a better idea for how to format the spreadsheet, you actually can edit the spreadsheet above and the changes will be saved. That’s pretty cool. Google Spreadsheets also can grant only specific people permission for editing.
So, does this approach work for groups to create and share standard fields in an effort to come up with widely adopted standard fields? One of the core objectives is to get as many fields commonly searchable across all MLSs as possible. Is that 50 fields? 100? More? If there aren’t many current fields, will MLSs be willing to change their current fields to adopt to new standards? Lots of questions here but I know there are many MLSs grappling with these exact problems right now and so this is the time to put keyboard to spreadsheet and define as many standard fields as possible.
Those reading the FBS Blog over the last few years will know that MLS data standards has been one of most frequent topics for my posts. I also spent a couple of years trying to lead the Real Estate Standards Organization (RESO) in that direction, with little effect. Thus, I went to the RESO meetings in Chicago this week with a lot of trepidation and even reservation that it would all be more of the same: no progress on data standards.
Indeed, there was a lot of the same discussion about why data standards will never happen (too much legacy data that will be lost, too expensive, too little incentive). Despite these perennial objections, however, there seems to be some good movement in the direction of standardizing MLS data. First, the Cove Group (a group of the biggest MLSs in the country) has decided that it would like to tackle data standards for their group. The members of the Cove Group are going to work among themselves to come up with their standard fields and then submit those into RESO as a change proposal to see if that can kick-start data standards.
This is a great step in the right direction, and I look forward to seeing what they develop. In fact, what I’d love to see happen is for the effort by the Cove Group to be open enough that other regional efforts already being undertaken can proceed at the same time with the same process so the results can be more easily coordinated into RETS. There are two simple things that would help achieve this objective: (1) RESO or the Cove Group should publicize the format in which they are going to publish the data standards once their work is complete; and (2) all groups should try to do their work out in the open on the web as much as possible so we can all track progress together.
During the meetings the last two days in Chicago, both of these topics were discussed a bit. First, the Cove Group representatives indicated they’d be using spreadsheets to collect, review and publish their standard fields. Second, yesterday RESO unveiled a new document management tool that will be used by the community to publish, review and comment on the RETS documentation. Currently, the document management tool isn’t yet live and so there isn’t any publicly accessible content, but, once live, all the docs will be available on the web. Also, currently the tool is primarily focused on the RETS specs, which contain little in the way of data standards, which often are best communicated in the form of a spreadsheet. Accordingly, I recommended to Chris McKeever and CRT that the tool be supplemented with the ability to include spreadsheets.
If the on-going work product of various regional groups could be published on the RESO web site through the document management tool, the likelihood that the efforts could be rolled together at the end would be much higher. Better yet, the spreadsheet formats each regional group is using should be the same. I know these are some basic issues, but I really think they’re important to making sure that the work of the various groups actually makes it into a cohesive national standard. For example, having the column headers the same so that the spreadsheets can be sorted easily in the same manner would be very helpful. In particular, how enumerations for fields are handled in terms of relationship to the primary field for easy viewing and sorting is important. Also, having a column identifying the source (regional effort or MLS) for the suggested field would be important.
Without some of these basic efforts at coordination and openness, there is a real risk that various groups will get pissy (oops, I mean proprietary) about their own work versus the work of the other groups. In contrast, if all the groups are working openly and in the same way toward the same goals, the result should be easier to coordinate. One easy way to coordinate and publicize the efforts would be to use Google or Zoho spreadsheets framed in RESO’s new document management tool. Here’s an example of how a Google spreadhseet can be framed in Confluence, which is the tool chosen by RESO for the document management function.
I fully understand that a lot of this data standards work will begin using Excel or other off-line spreadsheets, which is fine, but I’m strongly advocating that the off-line work be published for the entire community at as frequent of milestones as possible. The more transparent and open the work, the more people will be excited about participating with their own local/regional efforts.
So, that’s my pitch: openness and transparency as much as possible, please. The regional data standards efforts undergoing right now are fantastic and it would be even better to organize them into a cohesive national effort that help all the regional efforts work faster and together.
In a post last week, I asked a pretty open-ended question: “Are the costs of an MLS [coverting to a standard format] worth the long-term benefits?” I asked this question because we have several MLSs contemplating this very question now, and I thought it might be interesting for them to hear some outside perspectives. We received some great comments on that post, and in this post I want to put some more meat on the bone by outlining in more detail the potential cost savings for regionalizing MLS data.
Importantly, there are several ways to regionalize MLS data and so I think the first requirement is defining the objectives (benefits) of regionalizing. Once the benefits are quantified, then the costs of the potential solutions can better be judged and the “bottom line” or ROI of the entire effort will hopefully be clearer.
Some of the oft-sited reasons for regionalization are:
The next step is to quantify these costs or opportunities.
| Cost of Duplicate Listing Entry | |
|---|---|
| Formula | # Duplicate Listings Per Year x Cost Per Listing |
| #Duplicate Listings | Finding the number of duplicate listings is not always easy. One of the biggest challenges is that the data likely is not stored the same way for each of the MLSs involved, and so comparing addresses or parcel numbers may not be accurate. Also, you may not have ready access to the listing data from all of the other MLSs involved.Importantly, though, we’re not looking for perfection here but an estimate. In that regard, if you don’t have access to all the data for such an analysis or the address matching is too unreliable, an alternative approach would be to survey your brokerages/offices to ask them how many MLSs to which they belong and to estimate the percentage of their listings they enter into other MLSs. (This same info will be helpful in assessing duplicate membership fees discussed below.)For example, let’s say your MLS has 200 offices and ten percent (20 of them) belong to more than your MLS and have a policy of entering all their listings into those MLSs. Of those twenty, perhaps your survey finds that twelve offices enter listings into three additional MLSs, four of the offices enter into two additional MLSs, and four others enter their listings into just one additional MLS. You could then take the annual new listing count for each of those 20 offices and multiply it by the number of additional MLSs revealed from the survey for that office and you’d have a rough estimate of the number of duplicate entries for a year. |
| Cost Per Listing | There are a couple of methods for estimating the cost for entering a duplicate listing. First, you could use the cost for data entry personnel and an average amount of time for entering a listing as an estimate of the cost. For example, an average data entry salary is approximately $30,000 per year or about $15 per hour (rounding up). Assuming someone can enter a listing and upload the photos in 30 minutes on average (our time estimates are less than this, but 30 minutes is a conservative high-side estimate), the cost per duplicate listing is $7.50. On the other hand, many agents enter their own listings and so the cost really is the opportunity cost to them of their time, which is much more difficult to assess. For this reason, I’d recommend using the outsourcing cost as an estimate. However, I’d also add some additional cost for review by the listing agent and other compliance issues that arise. Accordingly, perhaps doubling the cost of listing entry is a good estimate for the cost of entering a duplicate listing. (Note: Of course, these costs may differ for you locally depending on relative salary costs in your area and other compliance requirements.) |
| Example | 1,000 duplicate listings entered per year x $15 per listing = $15,000 potential cost savings per year
Caveat: I’m just making up these examples. You need to put in your own numbers for your MLS. |
| Duplicate Membership Fees | |
|---|---|
| Formula | # Duplicate MLS Memberships x Average Cost Per Membership |
| # Duplicate MLS Memberships | If each MLS uses the NRDS ID as a tracking identifier for members, finding duplicates should be relatively straight-forward if you have access to the data. Other possible matching methods are email address or, as a last resort, name and possibly street address. Or, as mentioned above, you can get an estimate of the number of extra MLSs to which members belong by surveying your membership. Again, a solid estimate is what we’re after.
In designing your survey, you should be clear that you’re asking about MLS memberships other than yours. Or, alternatively, ask for total MLS memberships and then be sure to subtract one when doing your calculations. Another complicating matter here is that members may belong to multiple MLSs for a variety of reasons unrelated to data sharing or regionalization. For example, lock box access often is an issue that needs to be solve simultaneously with MLS data exchange. Overall, however, estimating the total number of duplicate members will provide an outside estimate of the potential cost savings. |
| Cost Per Membership | The cost per membership likely will need to be an average as calculating the exact number for each person would likely be impractical. |
| Example | 50 duplicate memberships x $240 average cost per membership per year = $12,000 potential savings per year.
Note: An interesting side note here is that the saved duplicate membership fees are lost revenue to the other MLSs in the region. The members save by not having to pay duplicate fees but the MLSs actually lose some revenue. |
| Costs of Maintaining Disparate IDX and Other Data Feeds | |
|---|---|
| Formula | (Total # of Data Feeds Delivered by All MLSs in Region – # of different entities receiving the feeds) x Average Cost to Maintain Each Feed
plus (Total # of New Feeds Delivered by All MLSs in Region – # of different entities receiving the feeds) x Average Cost to Setup Each Feed x Discount Rate |
| Total # of Feeds Delivered | This number should be relatively easy to obtain by looking at the RETS manager in your MLS system or inquiring with each MLS vendor involved.The number of new feeds delivered each year could be estimated by looking at the number of new requests received in the last year, and using that as a proxy or estimate for the number expected in the future. |
| # of Different Entities Receiving Feeds | You need to subtract the number of developers receiving feeds from the total number of feeds, because each developer would have to convert and maintain at least one feed no matter what. Note: Subtracting this number will only provide an average, because some developers may receive more than one feed from the same MLS, but, as mentioned above, we’re estimating here and this should provide a close proxy for the total extra feeds. |
| Cost of Converting and Maintaining Disparate Feeds | As an MLS vendor, we’re not on the receiving side of too many IDX data feeds, but we do have quite a bit of experience with data conversions. In a typical conversion, you have both programming and QA personnel involved and these processes collectively may take approximately 24 hours per property type to convert and test. Assuming an IDX feed would take about a third as much time (or 8 hours) and that an average MLS has 5 property types, that would be 40 hours per feed to get it set up. Assuming an average cost of $133 per hour for programming and QA, the cost to set up an IDX feed is about $5,320. (If there is anyone reading who has more exact cost estimates for converting IDX feeds than this, please comment or send me an email.)Of course, the conversion expense is a one-time cost, and, for those vendors already operating in your markets, the cost has already been incurred, and so will not be saved except for new feeds. The only cost that will be on-going is the cost to maintain the disparate feeds. Once the conversion program is written, the cost of maintenance is relatively minimal unless the MLS changes fields frequently, which is unusual. I would think a conservative (high) estimate of the cost to maintain a feed per year would be about $1,000 (7.5 hours per year at $133 per hour average for development and QA). |
| Discount Rate | A potentially important issue here is that the costs for processing the feeds often are not born directly by brokers or agents but rather by IDX and other product vendors. Accordingly, even if the process is made more efficient for these vendors, that doesn’t mean the prices paid by the brokers and agents will be lower. Of course, some brokers and franchises also hire developers in-house to process these feeds and competition likely will force prices lower over time, but it may be prudent to build in a “discount” rate on this potential cost savings to reflect that the savings may not all pass through to your members. |
| Example | (50 new feeds per year – 20 different entities receiving the feeds) x $5,320 per feed = $159,600 per year
(200 totals feeds – 50 different brokers receiving the feeds) x $1,000 per year per feed for mainteance = $150,000 per year As mentioned above, some discount rate likely should be applied to these formulas, because all the cost savings may not be passed through to your members. |
| Increase Exposure of Listings; Potentially More Sales | |
|---|---|
| Formula | Even though this is one of the biggest potential benefits, I’m not sure it can be estimated properly. First, there is at least some argument that no increase in sales will occur because the listings that benefit from exposure in more than one MLS already are being exposed by the duplicate entry. Second, if there are some people who aren’t willing to spend the time or money on duplicate entry now such that there would be some additional or faster sales from more exposure, estimating that number would be pretty hard. Perhaps one method would be to try to find out how many inter-MLS sales there are now on the duplicate entries and extrapolate that number to the non-duplicate listing base. However, such an extrapolation is filled with potential inaccuracy given that the agents and brokers likely weren’t entering it into the other MLSs already because they didn’t see a big benefit to doing so. Overall, I think this is the area where many regionalization decisions can go awry. The potential benefit of the extra exposure and sales seems limitless and so justifies nearly any cost when, in fact, the benefit may be negligible at best. |
The above are some of the key cost savings and efficiencies that can be gained by regionalizing or standardizing the data processing for an MLS. Perhaps there are others I’ve missed or different ways to estimate the savings. Please let me know in the comments. Also, if you have numbers for your area you’d like to share as real-world examples instead of my made up examples, that would be very interesting.
Once you have an estimate on the annual savings, you can then begin to compare those savings to the cost of getting there. The cost, who pays, and how likely the solution is to produce the savings identified above will depend on the strategy you choose for harmonizing the data. I’ll be addressing these different approaches in a follow-up post, so stay tuned. When we’re done, we should have a decent model for MLSs to assess the costs and benefits (or the “bottom line”) of regionalizing their MLS data formats.
The RETS community has been busy for the last year or so trying to prepare MLSs for June 2009, when the NAR MLS Committee has required all MLSs to be RETS compliant. This is a great step forward for national RETS compliance, but I have a more incentive-based approach for promoting RETS adoption.
NRDS is the National REALTORS Database System or, more simply, the directory for all NAR members. All the NAR-affiliated Associations of REALTORS are required to keep this system up to date. For many smaller Associations (the ones NAR is trying to get to adopt RETS), keeping NRDS up to date means double-entering the data into both the MLS system and the NRDS system. So, I’m thinking NAR could save everyone a lot of time and money by making NRDS RETS-compliant. Associations would win, NAR would win, MLS vendors would win, and NAR would win. That’s a lot of winning, from what could/should be a relatively straight-forward project for NAR.
