Killing the ‘old school’ dinosaurs with data-driven real estate platform content curation, recommendation and search engines, real estate investment with arbritrage sniffing. Bringing CTOs, data scientist, and deep learning guys to old school construction and real estate business.
Alternative download link: https://www.dropbox.com/s/ea2y8zjque3qw12/realEstate_dataDrivenAnalysis.pdf?dl=0
1. Data driven Real Estate-
Petteri Teikari P, hD
petteri-teikari.com
uk.linkedin.com in petteriteikari/ /
version Fri 13 January 2017
Killing the ‘old school’ dinosaurs
3. METADATA Structuring the text as well
Residence Features
● Spacious floor plans
● Direct elevator access with private and semi-private foyers
leading into each residence
● Double-door entry opening to expansive views
● Floor-to-ceiling high-impact windows with sliding glass doors
● Generous balconies with glass railings
● Ocean, city and Intracoastal Waterway views
● Large master suites with stunning views
● Walk-in closets in master bedrooms
● Freestanding soaking tubs
● Toto® bidet toilet
● Gourmet kitchens with chef's island and custom-designed Italian
cabinetry
● Stone countertops in kitchens and bathrooms
● Míele® induction cooktop range and concealed dishwasher
● Míele® wine cooler
● Sub-Zero® refrigerator/freezer
● Full-size washers and dryers
https://london.craigslist.co.uk/reo/5938505327.html
NOW DESCRIPTION
as “freeform
text” → Structure
into database needed
so it becomes usable
http://textminingonline.com/getting-started-with-word2vec-and-glove
4. metadata Hook up with all the area data US
People with family the
most interested in near-
by schools
●
Why stop there, add transit
times, near-by gyms, cafes,
restaurants, and the liveability
index of the area
http://www.zillow.com/
https://maps.nyc.gov/crime/
CRIME STATISTICS
5. metadata Hook up with all the area data UK
Find Properly https://www.findproperly.co.uk/
Nice start for the bot-
based idea
Asking whether you
want to be within
15min bicycle ride of
your desired location
(e.g. Hoxton, and
Shoreditch Old Station)
And I can exclude areas
for example like
Whitechapel, Stepney
and Shadwell
6. metadata Hook up with all the area data UK #2
https://www.police.uk/apps/
7. UK Pricing History data from land registry
https://www.bsa.org.uk/statistics/mortgages-housing
8. UK Pricing rightmove and Zoopla
http://developer.zoopla.com/
http://www.rightmove.co.uk/data/
http://www.zoopla.co.uk/market/uk/
The Zed-Index is the average
property value in a given area
based on current Zoopla
Estimates. Learn more
9. UK Area Analysis What is going on in the city?
The Bartlett - Centre for Advanced Spatial Analysis (CASA)
http://blogs.casa.ucl.ac.uk/category/big-data/
Agent-based Modeling in Geographical Systems
The Full Stack: Tools & Processes for Urban Data Scientists
http://www.reades.com/2016/10/14/the-full-stack/
“Rob Kitchin opened with a talk to frame the workshop, highlighting
the history of city data (see his paper on which the talk is based).
We are witnessing a transformation from data-informed cities to
data-driven cities. Within these data streams we can include Big
Data, official data, sensors, drones and other sources. The sources
also include volunteered information such as social media,
mapping, and citizen science. Cities are becoming instrumented
and networked and the data is assembled through urban
informatics (focusing on interaction and visualisation) and urban
science (which focus on modelling and analysis)”
The workshop, which is part of the Programmable City project
(which is funded by the European Research Council),
10. GEOLOCATION Profile the users
Use profiling with user preferred locations as “Pinterest trending” for trend
analysis, compare tosentimentanalysisinquantstockmarketprediction
https://www.toptal.com/javascript/a-map-to-perfection-using-d3-js
-to-make-beautiful-web-maps
Map “Hot/COLD” areas
Where people are willing to move so can we
identify real estate arbitrage options, see
Proportunity from London, UK for example
In other words constructing this map before
you see the increased demand in prices →
and not just retrospectively
http://www.telegraph.co.uk/property/house-prices/which-five-lon
don-boroughs-are-actually-in-demand-for-homebuyers/
11. Profiling actual fine-graining
London's population has soared over the past 35 years from 6.6 million in 1981 to 8.7 million
today, an increase of 2.1 million. However this masks a far greater change than the headline
figure suggests as London has literally be transformed from a mostly white British city to a
multi-layered multi-cultural city where the population density graph now masks the reality of
London now being several cities over laid upon one another which whilst all being
influenced by capital flows out of the central London global money markets, nevertheless will
havetheirown housepricesbullmarketdrivingepicentres.
http://www.marketoracle.co.uk/Article53301.html
Is ethnicity really that
predictive, and how much
of the variance does it
explain?
Have Facebook-level profiling
of people and see where the
hipsters are going and driving
up the propery prices?
When Shoreditch and
Williamsburg are just so over
and cannot provide good
returns
http://uk.businessinsider.com/facebook-data-brokers-2016-12?r=US&IR=T
12. GEOAnalysis indirect signals
So for example in London, you could start
doing “sentiment analysis” from Facebook,
Instagram, Snapchat whatever and see where
the gentrification vectors are pointing at:
Where all the coffee shops are coming, where
are the craft beers, amount of unconverted
warehouses, illegal warehouse parties and
exclusive sex orgy parties happening, etc. and
Try to predict the market as in the quant stock
market trading by doing sentiment analysis for
the stock market and trying to find signals
from Twitter (Bollen et al. 2011, cited by 2,243 articles),
etc. that would predict the stock market
changes
http://www.forbes.com/sites/modeledbehavio
r/2012/05/23/richard-florida-is-wrong-abou
t-creative-cities/#edebf217e426
13. GEOAnalysis investment returns?
Central London probably keeps its value rather well, but is it really the best investment location for risk-seeking
investorswhowant higherreturns?London onlyfor moneylaundryinvestmentsornot?
For example Bristol gave highest returns last year
“Bristol, which is the fastest growing city over the last 12 months, saw growth over the
last 3 months slow to 2.6% from a recent high of 5.0% in May 2016. Prices in
Cambridge fell by 1% in the last quarter although over the lst 12 months prices are
7.1% higher.”
https://www.hometrack.com/uk/about-us/press-room/july-2016-hometrack-uk-cities-house-
price-index-figures-released/
https://www.hometrack.com/uk/about-us/press-room/july-2016-home
track-uk-cities-house-price-index-figures-released/
Chinese money making
Vancouver unlivable with
the most overvalued
housing prices (price bubble
about to burst)
Canadian real estate booming with a bubble about to burst
for example?
14. e.g. 15 m²
Only one
bathroom
& toilet
semantic segmentation
Quantitative data from the floor plan
●
How many bed rooms. How big
are they? How many bathrooms?
●
Make this all searchable fields, so
if homebuyer wants only see flats
with living rooms above 15 m² for
example
15. Image-based valuation Needs more fine-grained labeling
Modern STYLE
The definition of modern could be the best tracked by
scraping from Pinterest all the time which serves as a
good “trend tracker” and what people find desirable?
OUTDATED STYLE Amateur photoProfessional photo
EXIF classification helps but you can get crappy images
with a DSLR as well. Think how good photo → horrible flat in person, vs.
amateur photo → looks actually better in person affects people’s willingness
to buy that. Do they feel like conned if the marketing material is too good
compared to perceived quality?
Think of various dimensions that could be relevant for classification/regression
●
In other words http://www.tractable.io/ for assessing real estate with visual characteristics
being one dimension. Interests banks and insurance companies
Think of indirect labels as well. Do not directly influence the value but helps assessing the
value from images. For example in bathrooms, typically less clutter (cleaner images)
CLUTTERED
The extra stuff is
not fixed and does
not correlate with
actual value
Non-CLUTTERED
Easier to analyze
room with the most financial value
16. Additional data Training set for material detection
Floor material will affect valuation. Hard to do from images
●
Problems of course with good-looking but cheap-feeling materials that
will feel cheap in person
●
Or with shabby chic
●
Needs more context at coarse-level. For example warehouse conversion
needs to be assessed differently from more or less similar looking farm
house.
http://www.aliexpress.com/store/produc
t/Decoration-Building-materials-Polish
ed-Crystal-Full-body-Tile-3D-floor-til
es-Porcelain-Bathroom-kitchen-Non-slip
/1908302_32493044896.html
Ceramic tile that looks like marble
https://www.alibaba.com/product-detail/c
eramic-tile-that-looks-like-marble_17551
28905.html
17. Image-based valuation in practice
The labelling tool need to accommodate “multi-pass” labeling
●
In practice could mean in practice:
1) Naïve annotator does the current coarse-level labeling (bedroom, living room, outliers, etc.)
2) The same person most likely have no idea what makes a flat valuable for Finnish / NYC market,
and we need some real estate agent doing that which needs an interface displaying the
metadata (location, etc.) for that person and that expert just gives the price estimate and nothing
else.
- The cost of this labeling differs, so hard to get huge volumes of valuation estimates.
3) For trends, we need to add the temporal dimension also for labels as over time tastes change,
Integrate the valuation part with Pinterest and interior design magazines/blogs (need to hand-
pick those that one thinks that is actually driving the taste of people rather reflecting it)
Think how this fits with the future, for example “quantreal estate investmenttools”
Slide 129
18. QUANTITATIVE DATABASE
●
Now we have the image database connected with the metadata
– City, neighborhood, address, elevation, size of different rooms,
number of bathrooms, internet speed, energy efficiency, near-
by services such as schools and their rating, NHS healthcare
trust in London fitted for your health condition, etc. other
custom requests that someone might have and what can pull
from open or closed data repositories)
●
This allows good queries to be done from this with bunch of
parameters.
●
And also we can generate image based on the metadata. E.g.
generate the “average” bedroom from a condo in Williamsburg
with this price, or something more useful.
19. Automated home-buying USA Style
fee of 6%, similar to the standard real estate commission,
plus an additional fee that varies with its assessment of
the riskiness of the transaction and brings the total
charge to an average of 8%. It then makes fixes
recommended by inspectors and tries to sell the homes
for a small premium.
20. Multimodal Quantitative model for real estate investment
AREA Analysis
- Schools, services, commute,
crime, construction projects,
noise levels,NHS, etc.
Sentiment analysis
- Facebook/Instagram etc.
- “Geoscraping”trends
VISUAL analysis
- Image Recognition
- Visual-based valuation
- Quantifyfloorplans
MARKET ANALYSIS
- Investwhere and tio what?
China, London, NYC, etc.
REGRESSION
forinvestment arbitrage
Same framework couldbe
hookedto a real estate
platformlike Zoopla and
Zillow as well, andfor
example provide only area
analysis
Zoopla could forexample do
“Geoscraping” via
FindProperly and function
as adataprovider forreal
estate investment brokers
(for bigmoney)
21. Image data without associated metadata
Problem now: Low-resolution texture (and
point cloud) with low-cost devices, how about
upsample the texture (and bump map) so that it
looks “cooler” than the pixelated original
Color map super-resolution:
Possible to learn the “model” of interior
spaces and how they should be upsampled
for optimized visualization.
http://blog.digitaltutors.com/understanding-difference-texture-maps/
https://arxiv.org/abs/1609.04802
Authorsfromex-MagicPony,currently TwitterCortexVx
Whattodowith “Real EstateImageNet” even without
the realestate specific data?
22. Super-resolution Texture Examples
https://papers.nips.cc/paper/2381-a-sampled-texture-prior-for-image-supe
r-resolution.pdf
https://arxiv.org/abs/1612.07919
Just emerging from semi-stealth mode (and even then, only barely), Magic Pony
Technology’s researchers have trained their system by exposing it to high- and low-
resolution versions of images and video, letting it learn the differences between
the two. MIT Tech Review was first with the story.
https://techcrunch.com/2016/04/14/magic-ponys-neural-network-dreams-up-new-i
magery-to-expand-an-existing-picture/
https://techcrunch.com/2016/06/20/twitter-is-buying-magic-pony-te
chnology-which-uses-neural-networks-to-improve-images/
24. Interactive immersive HALLUCINations
https://www.youtube.com/watch?v=9c4z6YsBGQ0&feature=youtu.be
Project: https://people.eecs.berkeley.edu/~jun...
Github: https://github.com/junyanz/iGAN.
When you have the semantic 3D model of your
flat, you could just generate the textures using for
example Finnish modern priors, and then shift to
outdated African priors in real-time when being
immersed to that space in virtual reality.
With the image dataset, one could generate the
textures then (well color map at least)
Hood believes VR will trickle down to lower-priced homes for
everyday sales within five years. He sees the floodgates
opening once the Zillow of VR is created.
“A number of young tech companies are exploring an entirely
in-VR experience where you enter search criteria like price,
location, and number of rooms and you’re presented a number
of homes and you can virtually tour,” Hood says. “Once that
happens, you’ll look back and say, ‘How did we do this
before?’”
http://fortune.com/2015/09/09/virtual-reality-real-estate/
25. HIRING Tech ‘buzz’ and branding
So with the real-estate related matierial you could train an autoencoder {or a generative model (like GAN)}
and apply that some video material and make cool promotional models/video just for fun and get into cool
lawsuits like Terence Broad with his Blade Runner
http://www.vox.com/2016/6/1/11787262/blade-runner-neural-network-encodin
27. Point Cloud Surveying resistance from RICS.org (UK)
Old-schoolsurveyors prefer their pen, ruler andtracingpaper
http://www.savills.co.uk/blog/article/192905/commercial-property/point-c
loud-data-capturer-to-help-smarter-surveying-practice.aspx
http://www.cbre.co.uk/uk-en/services/buildingconsultancy
/building_measurement
28. Point Cloud Resistance to technology from surveyors
http://www.frankham.com/service/surveying-project-management/measured-building-laser-scanning/
http://kykloud.com/the-construction-property-real-estate-technology-stack-where-do-we-fit-in/
http://kykloud.com/the-construction-property-real-estate-technology-stack-where-do-we-fit-in/
http://www.goreport.com/wp-content/upload
s/2015/12/whitepaper-mobile-future-of-bui
lding-surveying.pdf
http://www.sisv.org.sg/
29. PropTech killing the dinosaurs in UK
her-area-where-london-is-leading-the-world-a3348461.html
30. PropTech killing the dinosaurs in Singapore
http://disruptproperty.com/blog/singapore-iot-fund-sensing-cities/
http://disruptproperty.com/blog/proptech-in-asia/
http://www.propertyportalwatch.com/6-signs-singapores-real-
estate-tech-scene-is-on-the-rise/
http://www.property-report.com/5-proptech-companies-you-need-to-know-about/
31. PropTech killing the dinosaurs in USA
http://www.nytimes.com/2010/11/14/fashi
on/14eklund.html
Helps being a
good-looking
Swedish ex-
gay pornstar
in real estate
http://www.inman.com/hacker17/
http://dx.doi.org/10.1073/pnas.1321202111
Investors prefer entrepreneurial
ventures pitched by attractive men
32. GEOAnalysis investment returns?
Central London probably keeps its value rather well, but is it really the best investment location for risk-seeking
investorswhowant higherreturns?London onlyfor moneylaundryinvestmentsornot?
For example Bristol gave highest returns last year
“Bristol, which is the fastest growing city over the last 12 months, saw growth over the
last 3 months slow to 2.6% from a recent high of 5.0% in May 2016. Prices in
Cambridge fell by 1% in the last quarter although over the lst 12 months prices are
7.1% higher.”
https://www.hometrack.com/uk/about-us/press-room/july-2016-hometrack-uk-cities-house-
price-index-figures-released/
https://www.hometrack.com/uk/about-us/press-room/july-2016-home
track-uk-cities-house-price-index-figures-released/
Chinese money making
Vancouver unlivable with
the most overvalued
housing prices (price bubble
about to burst)
Canadian real estate booming with a bubble about to burst
for example?
33. PropTech Very conservative industry still – a lot of opportunities for Meaning
https://youtu.be/hER0Qp6QJNU
https://www.estateagenttoday.co.uk/breaking-news/2017/1/purplebricks-
people-may-never-accept-technology-only-sales
http://uk.businessinsider.com/gocardless-cofounder-ma
tt-robinson-launches-proptech-startup-nested-2016-9
Europe's First
PropertyTech
VentureCapital
Firm
http://pilabs.co.uk/
Slow moving industryasmillennialsin the end are not buyingthatmuchproperty