Mint’s unrefreshing contracted web scrapingdevelopers, customer adventures, finance — by TDavid @ 7:11 am PST
 (2 votes, average: 5 out of 5) Loading ... |
When it comes to the Techcrunch40 $50,000 winner Mint contracting Yodlee’s web scraping to get information that’s not being given them by banks with permission, Gerald writes:
“Well, OK. Gotta get after it somehow. I’m not overly concerned HOW the sausage gets made. I just want to eat.”
Oh Gerald, you can’t really be serious, can you? So you’d enjoy the stolen TV the neighbor gave you even if you knew it was stolen? Web scraping is a violation of the TOS of most sites and considered very, very bad netiquette. Unless/until Mint stops using scraping, I’ll be passing on their service. I don’t care if they are the most promising new financial Web pooh point app on the planet, how much they may or may not be able to make managing our finances easier and so on, scraping without permission is unethical behavior.
If Mint — or in this case their contracted provider Yodlee — can’t or won’t get the information through legitimate means — IE. cutting deals with financial institutions and using APIs — why should I trust them with any of my username and passwords? If they can’t follow the Terms of Service of other sites, why should I believe they will follow their own privacy guidelines with my or your sensitive financial information? What else will they fudge in the effort to provide a useful service to us? Sure, they are using TRUSTe and probably keep an otherwise clean kitchen, but this is a corrosive detail.
I covered why screenscraping is bad from a developer perspective over two years ago. My feelings on scraping, if anything, have hardened on the subject after continuing to see web app after web app acting like scraping is some kind of reliable business model or the right thing to do. And in a financial application where trust all around is essential, scraping is the rock chip in the windshield, soon to spider unless filled.
ProbargainHunter is right to label scraping “shady”:
This situation probably puts Mint in very uncomfortable position at very inconvenient time. It is ironic how life of a startup can depend on such a seemingly small thing. Web scraping has always been a shady business and I am surprised that Yodlee has gone with it so far.
Sorry for the cliche but just because somebody can doesn’t mean they should. Mint’s freshness, for me at least, has gone stale. Maybe I’m in the minority caring about these kinds of details, but so be it. I don’t just blindly do things because they are popular and/or provide me some benefit.
Related Posts- Unofficial bloggers breath mint: Myntz
- How dependable are you?
- Will the Simpsons movie in 2007 sink Titanic box office record?
- Y! Music has DRM flaw, says CIS
- Spam still getting through filters
- Why screenscraping is a bad idea
What??!! They use a scraper? Geeez… You could really get into trouble with that. Seriously. Good thing I haven’t given them any of my accounts. It’s a great idea, but I thought with all the VC funding they got, they had APIs from banks. Ick.
Not good. And to think I was actually thinking they were going to play a big role in the personal finance department. Not very cool. I’d agree there. Banks could seriously throw a wrench in their app without an actual business model. I’ve know cell phone apps (via web) that have gone to the wayside because of similar issues with web scraping.
I don’t think you’re wrong at all to think it’s bad. That’s a REALLY bad business decision to release information that is gotten from web scraping. Not to mention you could seriously insert some nasty stuff if scraped the wrong areas.
Comment by darkmoon — September 28, 2007 @ 7:52 am PST
If it’s against the T’s & C’s… It’s clearly wrong. No different than my stance on the Leopard NDAs I have in place. If, on the other hand, they’ve partnered with the banks… Another matter entirely. I’ll ask Damon and see where all this lands and reserve further opinion until I have some knowledge in hand.
GB
Comment by Gerald Buckley — September 28, 2007 @ 7:54 am PST
Good response, Gerald, please keep me in the loop on this one. darkmoon, good to hear you’re in the same camp
Comment by TDavid — September 28, 2007 @ 8:24 am PST
I wanted to provide a few comments on the Yodlee aggregation engine. “Screen scraping” is just one of the ways Yodlee brings a consumer’s data into a Yodlee-powered application, but a very large amount of data collected by a Yodlee application comes in through other structured feeds which are not websites. The use of a scraping technology doesn’t imply it’s being done without another institutions permission.
Many large institutions actually want data to be collected directly from a consumer-facing website, because the company has already made the investment internally to interface the consumer website with back-end financial networks or mainframe systems. Companies also have robust monitoring and load capabilities on existing web farms which are sitting idle through the night. It’s not always the right decision for them to build a whole new external interface into core banking platforms.
Comment by Jordan — September 29, 2007 @ 11:59 pm PST
Hi Jordan - thanks for stopping by. You wrote: “The use of a scraping technology doesn’t imply it’s being done without another institutions permission.”
I agree. The problem is you use the word “many” not all in the next sentence. All institutions giving permission is different than some, many or most. I’m glad you are here to set the record straight on the specifics.
Are you saying that Yodlee has the permission of every institution it is scraping from, yes or no? It’s really that simple. If the answer is no with even one institution, then permission needs to be obtained or the practice stopped. If the answer is “don’t know” then go back and find somebody at Yodlee who can answer that question. If the answer is YES, then there is no problem and was covered conditionally in the bolded text in the post above that reads:
“scraping without permission is unethical behavior.”
Scraping with permission = no problem. So, please answer the question. All institutions giving Yodlee permission? Yes or no?
Comment by TDavid — September 30, 2007 @ 6:40 am PST
My issue with this is…. if they use scrapes, then they’re logging in as the account owner and not through an API. Thus, if you get hacked, guess where the finger points? End user. From a banking perspective, that’s a huge pita for the end user, not to mention a pita for the bank. Screen scraping is also one big screw up if you the bank happens to change their design. What happens if the scraping company isn’t informed? It’s not like an API where you’re pulling the data fields direct.
Like I said before… having seen this done, this is not a safe measure for any end user and depending on how the scraping is done, you could actually introduce bogus data.
If it’s not being done via an API, scares the bejeebus outta me when you’re looking at banking and credit card information. Sounds like an ID theft disaster waiting to happen.
Comment by darkmoon — September 30, 2007 @ 8:04 am PST
Oh yeah, and I’d be amused to know if the scrapes go against the TOS for the banks.
Here’s Bank of America’s TOS:
“Using other aggregation sites. If you provide information about your Bank of America accounts to an aggregator company, we will consider that you have authorized all transactions initiated by an aggregation site using access information you provide, whether or not you were aware of a specific transaction.”
That’s pretty friggin scary. Basically, that says that if you’re allowing people into your account, then you’re authorizing them all access. I wouldn’t allow anyone full access into my bank account.
On top of that, scraping dies with two-factor authentication. Just try pulling a scrape through E-Trade when it’s got RSA’s SecurID. All in all from what I’ve read… all these people want the convenience of Yodlee. That’s fine, but in the world where ID theft is more common through electronic means, I move towards hardier methods of protecting the data. That would mean that scraping itself is not allowed and even API access should be through a secure method.
Comment by darkmoon — September 30, 2007 @ 8:14 am PST
“Are you saying that Yodlee has the permission of every institution it is scraping from, yes or no?”
While the question seems simple enough there’s plenty hidden implication in it that make it very difficult to answer.
If a bank has a TOS that allows its customers to aggregate their data from it, does that blanket permission constitute permission to Yodlee?
If the TOS is vague, like most, and says that customers have a right retrieve and use their data in third-party applications, without mentioning the means of getting that data, does that constitute permission to Yodlee?
If a financial institutions notices a high volume of traffic from the Yodlee servers, contacts Yodlee and accepts what is occurring and chooses not to block Yodlee traffic but doesn’t draft any official agreement to avoid lengthy legal processes, does that constitute permission?
I wish it were simple, but the complexity of law makes it a complex issue. People should weigh in their minds if they think it’s ethical to access their personal information through a service that might not have explicit legal agreements with all the financial institution that currently houses their data. If they feel it isn’t right, than they can choose not to use a Yodlee-powered service.
These types of discussions are always welcome on the Yodlee Forums:
http://forum.yodlee.com/
..Jordan, Yodlee Inc.
Comment by Jordan — October 6, 2007 @ 7:54 pm PST
You guys might want to check out spendview.com … after talking to the guys there through their meebo client which I thought was very helpful and smart. I think they mentioned that they are using only OFX standard to download transactions. They said they don’t support scrapping data from Financial Institutions that don’t allow OFX download but you can manually upload the data. I found out the sign up code through a finance blog comment tip (don’t exactly remember where??). Anyways, if you go to their tour page, at the bottom of the image there is a link that you can follow to sign up for a free beta account.
Comment by Bruce Ken — October 16, 2007 @ 9:02 pm PST
[…] Scraping without permission is wrong. Sorry, yes, even scraping your own data from a third party site. […]
Pingback by Scoble breaks Facebook TOS in Robin Hood data portability effort » Make You Go Hmm — January 3, 2008 @ 10:25 am PST
I believe even Quickbooks Online does “scrapping.” They had problems earlier this year trying to get our Bank of America account to work with downloading automatically data from BofA.com to Quickbooksonline.com every night. Their IT support told me that their programmers were working on it and that it has to do with how BofA’s web site worked and that BofA changed some things on the site that stopped Quickbooks from logging in and getting my data (and others - every BofA customer with QuickbooksOnline had this problem I was told). So I’m assuming Quickbooks does the same thing that Yodlee does.
Comment by Andrew — January 6, 2008 @ 12:57 pm PST