Here are some tips from Adam Bosworth (who worked on ODBC, XML, etc.) on how to build a successful standard:
1. Keep the standard as simple and stupid as possible.
2. The data being exchanged should be human readable and easy to understand.
3. Standards work best when they are focused.
4. Standards should have precise encodings.
5. Always have real implementations that are actually being used as part of design of any standard.
6. Put in hysteresis for the unexpected.
7. Make the spec itself free, public on the web, and include lots of simple examples on the web site.
To all those involved in trying to create RETS data (not transport) standards, how many of these have we followed?
(h/t Joel Spolsky)
The Real Estate Standards Organization (RESO) announced the results of the Board of Directors election this last week, and I’m very happy to report that Troy Davisson from FBS was elected to a two-year term. Troy has been a tireless advocate of RETS and contributed to many work groups over the years. Troy brings both server and client developer perspectives to the Board and will make a great contribution.
Congratulations also to Mary Frances Adams (Trend MLS), Paula O’Brien (consultant), and Steve Clarke (Marketlinx) on being elected, and a big shout-out to Kristen Carr (Bridge) for being the first RESO director to be re-elected. Here’s the entire RESO Board for 2010:
Mary Frances Adams – New
Pat Bybee
Kristen Carr – Reelected
Steve Clarke – New
Troy Davisson – New
Sergio DelRio
John Holley
Mark Lesswing
Chris McKeever
Paula O’Brien – New
Paul Stusiak
Mark Wise
I’m hopeful this Board is able to leverage the energy in the community around data standards and prioritize this work to provide a path for MLSs wanting to standardize. I’ve mentioned this many times, but there are efforts going on in many parts of the country and all these efforts would greatly benefit from some standards to start work from and point toward.
I’ll be moderating the MLS panels at Inman Connect NY in January, One of the sessions outlined so far is “Breaking Data Taboos for Fun and Profit.” The idea for this session stemmed from the Connect Create session at Inman SF this last summer, where developers from two companies built two applications in 48 hours and then showed them off at the end of the conference. One of those applications was an agent rating service built by Diverse Solutions using some data from SoCal MLS.
During the demo of that product, Diverse’s President Justin Lajoie commented that he wasn’t sure if the product would ever see the light of day because MLSs would have to grant permission to use the sold data in this way. Brian Boero asked the brokers and agents in the audience if this was a product they’d like to see become real, and the response was definite: Yes, they would.
The question raised was pretty clear: How can MLSs better leverage the kind of rapid innovation available today? One potential answer is to create an API for the MLS data that’s easy to use and understand, especially the terms of use. Currently, the only terms of use for MLS data that are widely adopted by MLSs are IDX and VOW policies. Are those enough? Or could brokers and MLSs create more innovation by developing a new terms of use targeted at specific data sets?
For example, two potential use cases come to mind: (1) syndication or advertising of listings; and (2) aggregate statistical reports. In the case of syndication, the terms of use would focus on the limited set of data needed for advertising the listing and would include an opt-in from the listing broker. Having a standard API for syndication could increase the quality of listing advertising on the web and increase competition among aggregation sites.
In the case of aggregate statistical reports, the terms of use could focus on limiting use of the data for analysis and reporting in the aggregate as opposed to disclosing individual listings. Would a more limited terms of use focused on the aggregate instead of specific listings make it less threatening to brokers to open up the data to new and innovative uses? Are there any terms of use that would be able to be widely adopted or will MLS data use always be limited to policies like IDX and VOW?
Of course, these discussions do not happen in a vacuum. Just yesterday, Google made it easier to see real estate listings on their maps and they’re encouraging real estate professionals to post their listings to Google. Of course, Google has their own terms of use for posting information to their site. We also know that companies like RealBird are using Google Base as an alternative source for listing data, using Google’s API as a round-about way to get at the MLS data.
Is it time for MLSs to leverage the rapid innovation cycle by creating their own API? I’d love to hear from you in the comments below and also at Inman Connect NY.
I’m very excited to announce that the Tucson Association of REALTORS® MLS (TAR/MLS), representing approximately 6,000 members in Ariziona, has selected FBS and our flexmls Web System for their next MLS system. The TAR/MLS Committee and Board of Directors engaged in a very thorough review of the major MLS vendors and concluded unanimously earlier this week that FBS and the flexmls Web system was the best company and system for them.
From a company perspective, getting the opportunity to work with the staff and members at TAR/MLS is a huge win for FBS. TAR/MLS will be one of FBS’s largest customers and furthers our goal to provide great service to MLSs of all sizes. From a personal perspective, I’m excited to get to work with such an incredibly well-run organization. Getting to know and work with Wes Wiggins, Brian Case, Scott Weidamoyer, Andy Gordon and others at the MLS has been great and we’re all excited to get to work together on the conversion.
I’m also excited to get to work with the forward-thinking Board at TAR/MLS. I’d like to give a special shout-out to Director John Mijac who, with Wes Wiggins and staff, put in hundreds of hours (or more) leading the group evaluating the various vendors. I’ve gotten to know John pretty well over the last several months and found him to be exacting and exhaustive in his research and presentation skills. John left no stones unturned in his review but, if he had, they would have been uncovered by President Kim Clifton, who conducted one of the most detailed executive interviews I’ve ever been through. I left that meeting incredibly impressed with Kim, and plenty exhausted as well. Jim Adams from Long Realty also dug into our RETS services and related processes to ensure that we had the right tools to feed the back-office and other systems they and other brokers require. Overall, the entire Board was thorough and objective in their review and we learned a lot from their professional review process.
Getting to work with all these great people is what makes this business so rewarding. The months ahead with all the conversion work and introduction of a new system to the members of TAR/MLS undoubtedly will be made much easier because of the great people involved. All the employee-owners of FBS sincerely thank the leadership at TAR/MLS for trusting us with such a great responsibility, and we look forward to living up to your expectations!
Here’s an email I received today from a member of the Board of one of our MLS customers:
Hi Michael,
I believe you are involved with the Data Standards group at NAR, right? We are still struggling with some definitions and wondering if there is anything published by the data standards group on things that have already been agreed on at the NAR level. Do you have anything like that? Yesterday we struggled with “Single Family Detached” home for over an hour. Is there some help out there?
Thanks, and look forward to seeing you in San Diego.
I typically discuss data standards in relation to MLSs wanting to share data or regionalize, but this is a good example of an MLS simply wanting an easier way to get the data right. I pointed this customer to the RETS schema, but also had to say that it wasn’t user-friendly and didn’t say anything very specific about her question.
From my perspective, a broker asking for a definitive data definition to be published at the national level isn’t too much to ask. I would really like to be able to respond, Yes!, here’s a link to the data dictionary we’ve been working on for the last many years and the definition you’re looking for is on row 28 (or whatever). Wouldn’t that be great? Let’s make it happen.
Google announced yesterday that they are now sourcing a lot of their street mapping data from government sources. What wasn’t immediately clear from their announcement, though, is that they are no longer using TeleAtlas data for their base street maps in the U.S. Instead, apparently Google now owns their own base street maps, and is including parcel maps for many counties across the country.
It may not be immediately obvious, but this is a BIG deal to anyone who uses map data and it raises a ton of questions. What will the terms of Google’s licensing be now that they no longer have to pay TeleAtlas for the base map? Given that they are sourcing the data from many government sources, will Google be providing the data they get from users back to the counties? Will they contribute this base map to any open source efforts like Open Street Map?
I’m very curious what others have experienced in terms of pricing from Google lately for Enterprise Map Services. Our renewal was more than interesting, and it was shortly prior to this announcement that they are no longer using TeleAtlas. I’m really hoping Google is intending to live up to their “Do No Evil” motto, because there’s a ton of potential here for them to be building a huge base map with tons of data from government sources that they control.
In my last post, I mentioned that RETS data standards may get a boost from the COVE Group of MLSs starting to work together on further defining standard field names for their MLSs. Standard names have existed for a long time in RETS but they are rarely well populated. This new effort is to reinvigorate standard names and try to figure out how they can be changed so they are actually put into use.
Because there are other regional data standardization efforts across the country, I suggested in my last post that sharing the process for this work with the broader community would be helpful so that the work can be coordinated with work other groups are doing. One of the most basic first steps is defining a format for publishing the agreed upon fields. Spreadsheets tend to be the easiest way for the broadest group of people to access the results, and so they are most commonly used in data standardization efforts.
One of the challenges with spreadsheets, though, is that they are two dimensional and the listing data often is nested, especially when it comes to fields with a list of values. How do you best show the field name and the list of values? One approach is to combine them into one sheet and another is to include the list of values on a separate sheet as shown in the example below (though I couldn’t figure out how to link the two sheets together in Google documents).
Another question is whether to include the data type in this initial effort. In the past, defining data type has created a lot of consternation where, for example, some MLSs use a list of values for number of bedrooms while most others use a numeric value. If this effort is to have meaning and usefulness long term, I think we have to define data types as well as names. Lastly, even though RETS schema doesn’t define fields by property type, I suggest the spreadsheet contain a property type column to define those fields particular to certain property types.
I think using Google Documents or some other web based spreadsheet would make sharing and publishing the information on sites like RETS.org much easier for everyone. Sharing is critical for getting input and feedback, and building consensus around the results. For example, if you think of a better idea for how to format the spreadsheet, you actually can edit the spreadsheet above and the changes will be saved. That’s pretty cool. Google Spreadsheets also can grant only specific people permission for editing.
So, does this approach work for groups to create and share standard fields in an effort to come up with widely adopted standard fields? One of the core objectives is to get as many fields commonly searchable across all MLSs as possible. Is that 50 fields? 100? More? If there aren’t many current fields, will MLSs be willing to change their current fields to adopt to new standards? Lots of questions here but I know there are many MLSs grappling with these exact problems right now and so this is the time to put keyboard to spreadsheet and define as many standard fields as possible.
Those reading the FBS Blog over the last few years will know that MLS data standards has been one of most frequent topics for my posts. I also spent a couple of years trying to lead the Real Estate Standards Organization (RESO) in that direction, with little effect. Thus, I went to the RESO meetings in Chicago this week with a lot of trepidation and even reservation that it would all be more of the same: no progress on data standards.
Indeed, there was a lot of the same discussion about why data standards will never happen (too much legacy data that will be lost, too expensive, too little incentive). Despite these perennial objections, however, there seems to be some good movement in the direction of standardizing MLS data. First, the Cove Group (a group of the biggest MLSs in the country) has decided that it would like to tackle data standards for their group. The members of the Cove Group are going to work among themselves to come up with their standard fields and then submit those into RESO as a change proposal to see if that can kick-start data standards.
This is a great step in the right direction, and I look forward to seeing what they develop. In fact, what I’d love to see happen is for the effort by the Cove Group to be open enough that other regional efforts already being undertaken can proceed at the same time with the same process so the results can be more easily coordinated into RETS. There are two simple things that would help achieve this objective: (1) RESO or the Cove Group should publicize the format in which they are going to publish the data standards once their work is complete; and (2) all groups should try to do their work out in the open on the web as much as possible so we can all track progress together.
During the meetings the last two days in Chicago, both of these topics were discussed a bit. First, the Cove Group representatives indicated they’d be using spreadsheets to collect, review and publish their standard fields. Second, yesterday RESO unveiled a new document management tool that will be used by the community to publish, review and comment on the RETS documentation. Currently, the document management tool isn’t yet live and so there isn’t any publicly accessible content, but, once live, all the docs will be available on the web. Also, currently the tool is primarily focused on the RETS specs, which contain little in the way of data standards, which often are best communicated in the form of a spreadsheet. Accordingly, I recommended to Chris McKeever and CRT that the tool be supplemented with the ability to include spreadsheets.
If the on-going work product of various regional groups could be published on the RESO web site through the document management tool, the likelihood that the efforts could be rolled together at the end would be much higher. Better yet, the spreadsheet formats each regional group is using should be the same. I know these are some basic issues, but I really think they’re important to making sure that the work of the various groups actually makes it into a cohesive national standard. For example, having the column headers the same so that the spreadsheets can be sorted easily in the same manner would be very helpful. In particular, how enumerations for fields are handled in terms of relationship to the primary field for easy viewing and sorting is important. Also, having a column identifying the source (regional effort or MLS) for the suggested field would be important.
Without some of these basic efforts at coordination and openness, there is a real risk that various groups will get pissy (oops, I mean proprietary) about their own work versus the work of the other groups. In contrast, if all the groups are working openly and in the same way toward the same goals, the result should be easier to coordinate. One easy way to coordinate and publicize the efforts would be to use Google or Zoho spreadsheets framed in RESO’s new document management tool. Here’s an example of how a Google spreadhseet can be framed in Confluence, which is the tool chosen by RESO for the document management function.
I fully understand that a lot of this data standards work will begin using Excel or other off-line spreadsheets, which is fine, but I’m strongly advocating that the off-line work be published for the entire community at as frequent of milestones as possible. The more transparent and open the work, the more people will be excited about participating with their own local/regional efforts.
So, that’s my pitch: openness and transparency as much as possible, please. The regional data standards efforts undergoing right now are fantastic and it would be even better to organize them into a cohesive national effort that help all the regional efforts work faster and together.
There’s been a great discussion going on over at Bloodhound Blog regarding whether and when to ask visitors to an IDX site to register. In his post If You Want to Close More Deals, Require Registration, Eric Bramlett makes the case that requiring registration can produce more sales if you have a good follow-up process in place. The basic premise is that requiring registration produces more sign-ups from a wider variety of people (including those not wanting to buy or sell now), and so you need an incubation process to find those ready to do business now. In the comments to the post, there are arguments pro and con for requiring registration, with those who don’t require registration preferring to allow users the freedom to indicate when they want to register, which provides a sort of self-selecting process (fewer but higher quality registrations).
To help our IDX customers find the best options for their site, FBS recently released options in its IDX Manager to allow agents and brokers to specify registration options for each link:
There are a couple of things I think are cool about this: (1) the options can be set up for each link, so you can try different options on different links and compare how they perform; and (2) you can allow users to skip the registration requirement, which effectively turns the requirement into a request.
My personal view on registration is that I think it should be as natural to the consumer’s search activity as possible. For example, it’s necessary and expected for users to register when they want to save something, and that’s the default behavior of our IDX links. If you’re going to require a registration to view content, perhaps requiring for it right up front is the most natural — if you want in, identify yourself — because some arbitrary limit based on time or number of listings or searches viewed is, well, arbitrary.
Overall, as I’ve written before, I think consumers are more accustomed than ever to registering at web sites, and so asking for registration no longer has the negative connotation it once did. I also think registration requirements are going to become more common on real estate sites as VOWs gain popularity. As others have commented, the key is how you follow up. If you provide a useful service post-registration and don’t spam the recipients, the consumer may just become a valuable customer making the registration a win-win.
I also realize that what matters are measurable results. Some or all of these approaches may work better for you than others. That’s why we’ve provided the options, for you to try and see what works for you. As these new options are deployed by our customers we’ll collect data and report back later with which options are producing the most registrations and it will be interesting to see if we can tie the registration data to sales data to measure the real bottom line.
Let me know your thoughts on the best place to ask for or require registration during the consumer search process.
Posting here on the FBS Blog will be light or non-existent over the next two weeks as I’ll be on vacation, hopefully enjoying some boating, fishing and other lake fun at our cabin in central Minnesota.
