Is there a super plugin that pulls in your idx or rets data into WordPress that is free, open source and easy to use?

Over the last 11 months I’ve followed a dozen or so threads related to the subject of IDX RETS and WordPress and have noticed with ever increasing frequency that people expect a one size fits all application to actually exist.  Although there are plugins that will integrate IDX and RETS based on special circumstances there are no existing plug-ins with the capability of fitting the bill for all situations and in this post I wanted to point out the top five reasons why.

No two MLS sources are ever alike.

Thousands of MLS providers, hundreds of service professionals and dozens of software vendors have yet to come up with an easy to implement and globally standardized format. Given all the different types of IDX and RETS services provided by the various boards, I have yet to encounter where any two are ever alike beyond casual similarities. Below I’ll go over mapping and why that stands as one of the biggest hurdles but first I’ll say that to build a plug-in that can address all types of data feeds would not only be daunting, but extremely expensive to build, maintain and even manage.

Here are a few data feed types that such a plug-in would need to take into account.

  • Push type data feeds and these include time stamp and even encoded text and csv files that must be decoded before processing. Push means that the data provider will PUSH the data to a folder on your server.
  • Pull type data feeds which in addition to the above mentioned will also require scripting for FTP or even an FOPEN URL path to access the data. The limitation here is that the url file include method wont always work as many servers have this feature frequently disabled.
  • Media server types which are most often domain locked. This means that you may have a feed, push or pull but some have file and access points that are domain locked. Scripting
  • IP locking of access to the various feeds. This includes RETS feeds. Many RETS service providers will domain or IP lock your access to the data. A plugin would need to take into account settings for how it will access the data if it is to be useful in most production sites.
  • RETS versions and the differences are not backwards compatible all the claim is that they are. Go to RETS.org and the professionals there will quickly tell you that not all RETS versions are alike. The plugin would need to auto detect the type of RETS version being used and then adjust according to additional settings for actually getting the data. One of my favorite tools is the vieleRETS downloader which is as close as you can get to a globally compliant RETS downloader anywhere. The only issue is that without ideal setting, it can tank a server if not properly configured.
  • RETS and IDX are not at all the same. RETS uses an XML wrapper to shake hands and communicate as a sort of translator for lack of a better term between two different databases. In essence if you speak spanish and I speak english the person in the middle doing the translation for us is this thing called RETS and all that it embraces.
  • IDX can be in many forms including remote SQL connections to the downloading or pushing of data and photo archives which once extracted would be processed depending on your particular database structure.
  • I can go on and on but I’ll stop here by saying its not realistic to believe such a solution could exist at this point.

Hosting for real estate is a unique animal.

Hosting for real estate websites is a real different animal for sure. There are configuration settings required for real estate hosting that almost none of the commercial shared solutions can accommodate. The biggest hurdles in making a plugin that would be able to do all things would have to take into account things like step processing, offset downloads, file splitting, and a hole host of other little things that developers frequently encounter.  When it comes to actually processing the images and other data elements a server setting called the max_execution_time setting plays a role.  MLS scripts can and often will run for several minutes non stop during the database updates. Most servers are not configured beyond 30 seconds for this to take place.

Boards of Realtors Just Say NO!

I’ve encountered boards of Realtors that literally said no to a scripting tool that would be used by the actual website owner. In fact I’ve encountered some MLS providers that wont talk with anyone beyond programmers and IT professionals even if the person contracting the solution is the broker / owner of the company.  Some MLS’s are in the position that leaving such a tool available opens the door for abuse by novice users and would expose their data. I disagree but paranoia is common among many MLS’s still today and their entire policy is dictated by an inventory of precautions.

Compliance isn’t really a reason for not building such a plug-in but I’m calling it one.

First and foremost, if you don’t think the industry is full of hidden loyalties, underhanded competitors and snitches that seek to debunk your every effort you are deliberately indifferent to the truth.  The reason a one size fits all plug-in would be almost impossible to publish is quite literally someone would call foul ball.  Compliance is the number one reason real estate websites run into hot water. Compliance isn’t just about knowing which fields to and not to include as would be expected. Various elements of the data source among other things must be included in compliance disclosure and boards frown on anything that appears to buck the trend.

The issue of mapping the fields.

Last and in some ways the most important. If you have ever opened up an IDX data feed file and looked at how its structured you would likely notice its some sort of delimited structure possibly a CSV comma delimited type.  Mapping a CSV to a database isn’t rocket science but mapping with proper field concatenated values is essential for effective search and display features of the listings data.  In all my years of experience I’d say that mapping equates to 70% of the over all time in any RETS or IDX project. I’ve looked for ways numerous times to reduce this action to something a bit more manageable and have yet to find any that really work much better overall.

So in summary, due to the exhaustive differences between all the MLS data feed types, it would be nothing shy of a seriously expensive effort to develop such a plug-in.  But all hope isn’t lost since there are tools that can be adapted to fit most scenario.