real estate data

Protect Broker Data with Anti-Scraping

by Victor Lund on April 9, 2014

If you are a broker or an MLS, you must treat your data as an asset. By and large, our industry does a great job of protecting the front door of data access, but the back door of data scraping is wide open. It begs the question, if you care about data protection, why are you only scrutinizing the legitimate data users without hunting for the nefarious and illegitimate data thieves? Thieves that want to steal data send robots out onto the internet to do their pickpocketing, and they do it quite well. Reviewing Distil Networks’ reports with Matt Cohen of Clareity Consulting heightened my awareness of how clever these people are. The rookies are the ones who are attacking from outside of the country. The good ones attack on networks like Charter and Comcast and other common Internet service providers that cloak them. They even use simulated browsers to access your site so that it is difficult to detect automated behavior. Until recently, the answer was related to the high costs and high technical acumen that would need to be absorbed somewhere. Who can operate the solution and who will pay for it? But with Distil Networks, for a small to midsized broker with 50 website domains, the cost would be $100 per year to cover 1 million page requests a month. A single site is $36 per year. That is hardly an economic barrier and that includes support. Sidenote: Those ComScore numbers that many portals are bragging about are not all consumers – double-digit percentages of them are robots. One of the chief reasons why® traffic was outpaced so heavily by others is due to their implementation of anti-scraping solutions. By doing the right thing, they got their teeth kicked in. Protecting every site in an IDX region will only happen if you mandate it across all IDX vendors and the MLS itself. Many MLSs already charge data access fees. Perhaps they can use those fees to shelter the costs of an anti-scraping solution. The issue with any mandate or requirement of this type is that it would only be successful with the support of the participating brokers. Without that support, it would be a political nightmare for any MLS. Aside from blocking data scraping, products like Distil are effective at setting a trap to snare companies who are stealing and profiting from data scraping. The behavior […]


MLS Data Standards

by Victor Lund on March 17, 2014

MLSListings is the MLS service provider in the greater San Francisco Bay Area, which includes; Silicon Valley, San Jose, Monterey County, Santa Cruz County, South San Francisco, and parts of the East Bay. It is a large region in the heart of the most daunting techno-consumers, techno-brokers, and techno-agents in America. Satisfying the needs of this audience is intense, but MLSListings has been doing it successfully for many years. They are a company that is constantly evolving to meet the ever-expanding demands of real estate information in their market place. MLSListings CEO, James Harrison published an excellent paper last week called MLS Standards: Can and do Exist. You can read the paper below.  One of the strong take-away points from the paper is the plea that more MLSs join MLSListings in adopting and adhering to the Real Estate Standards Organization or RESO standards. Here is the ugly truth about the issue of standards:  All NAR affiliated MLSs are supposed to adhere to the RESO standard according to their charter. That includes the Real Estate Transaction Standard or RETS. In other words, all MLSs are under an agreement that requires them to be RESO and RETS compliant. NAR is in the difficult position of having to enforce standards, but it has been challenging. As a result, few MLSs have data standards in America today. There are 850 different data standards and systems that do not talk to each other unless someone maps the data and maintains the mapping. It’s insane for the brokers and agents who need this level of consistency for their business practices. Of the 850+ MLSs in America today, only “64 are charter members of RESO,” according to Harrison’s paper. This is evidence that many MLSs are simply asleep at the wheel. There is absolutely no reason why every MLS and every RETS system cannot be compliant with the RESO standard. MLSListings is a leader. It is an MLS that is way ahead in adopting the RESO standards. In truth, most of the large MLSs in the nation are part of RESO and are RETS compliant. I give a lot of credit to the COVE Group for making this happen, especially Anne Bailey for her role as the facilitator. Where is the rest of the industry? Whenever large firms bring this issue to their local board or MLS they get rebuffed as bullies. Who else can make a […]


Brokers harvesting business from their data

by Victor Lund on February 25, 2013

WAV Group White Paper - Broker Data

Real estate brokerages are way behind in the curve of excellence in lead management. This report is not about best practices of routing leads to agents, it is focused on measuring lead quality, and appending lead information with business intelligence that will convert more leads to drive more revenue. WAV Group’s brokerage business is focused on enterprise brokers, 500 agents or more – and usually a brokerage that also has some type of relationship with a mortgage or an insurance company. We use the term “home services” or “ancillary services” to refer to a brokerage that provides a full suite of residential, mortgage, and insurance services to the consumer through its enterprise business units. These companies also frequently have a complement of settlement services and home warranty services that may also be offered. Only brokerages that have significant size and offer a suite of home services will have enough data to profit from the harvesting practices outlined in this report. This report illustrates the best data analytic and business intelligences practices by the nation’s largest banks and mortgage companies. Using these examples, we explain how large real estate brokerages with home services offerings can become more competitive and more profitable from adopting these best practices. To access this report, please click this link.