courser web intelligence and big data 0 introduction lecture slides

Web
 Intelligence
 and
 Big
 Data
 
introduc3on
 to
 the
 topic
 
course
 outline
 and
 logis3cs
 

 


The
 RThe
 
everse
 
Turing
 
Turing
 
Test
 Test

 
Which
man,
and
Which
is
man,
and
Which
isismachine,
and
which
whichisisiswoman???
woman???
???
which
human
Conv
ersa
ti

on…


Like / Dislike
Shopper / Surfer
Rich / Poor
Ethnicity

The image cannot be displayed. Your computer may not have

The image cannot be displayed. Your computer may not have
enough
memory
to open
the image,Your
or the
image may
have been
Thetoimage
be displayed.
computer
enough memory
open cannot
the image,
or the image may
have may
corrupted.
your
computer,
and then
openor
the file again. If
notRestart
haveRestart
enough
memory
to
open
image,
been corrupted.

your computer,
and
thenthe
open
the filethe
the image
red x still
appears,
you may haveRestart
to delete the image and
may
have been
again. If the red
x still
appears,
you corrupted.
may have to deleteyour
the
then
insert it again.
computer,
then open the file again. If the red x still
image and then
insert it and
again.
appears, you may have to delete the image and then
insert it again.

The image cannot be displayed. Your computer may not have enough memory to open the image,
or the image may have been corrupted. Restart your computer, and then open the file again. If the

red x still appears, you may have to delete the image and then insert it again.

Original
‘Imitation’
Turing
Game
Test
Happening all
the time!

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted.
Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.


Web
 Intelligence:
 web-­‐scale
 AI
 is
 here
 


Big
 Data
 ?
 
Ø  lots
 and
 lots

 of
 web
 pages
 …
 
Ø  a
 billion
 Facebook
 users
 
Ø  billion+
 Facebook
 pages
 
Ø  hundreds
 of
 million
 TwiCer
 accounts
 
Ø  hundreds
 of
 million
 tweets
 per
 day
 
Ø  Billions
 of
 Google

 queries
 per
 day
 

Moore’s
 Law
 

Ø  Millions
 of
 servers,
 petabytes
 of
 data
 
In
 contrast,
 typical
 large
 enterprise:
 
q 5000-­‐50,000
 servers,
 
 
q Terabytes
 of
 data,
 millions

 of
 Txn/day
 

 

Kryder’s
 Law
 


Big-­‐Data
 technology
 
tradi3onal
 `business
 intelligence’
 using
 databases:
 

 

 

 
databases
 
data-­‐warehouse
 

sta3s3cs
 
more
 databases
 

 
Google,
 Facebook,
 Linkedin,
 eBay,
 Amazon
 …
 
did
 not
 use
 `tradi3onal’
 databases
 for
 `big
 data’
 
why?
 
 
what?
 
• 
  massive

 parallelism
 
 
• 
  Map-­‐Reduce
 paradigm
 

 


what
 does
 data
 have
 to
 do
 with
 intelligence?
 
“any
 fool
 can
 know
 …
 
 the
 point
 is
 to

 understand.”
 

-­‐
 Albert
 Einstein
 
 

and
 …
 the
 goal
 of
 understanding
 is
 to
 predict
 
Listen

Predict

Reac?ve
 Intelligence
 

Predic?ve
 Intelligence
 

web
 intelligence
 using
 big
 data
 
AI
 techniques
 at
 web-­‐scale
 for
 `predic3ve
 intelligence’
 

 
Ø online
 adver3sing
 –
 predic3ng
 intent
 and
 interest
 
Ø gauging
 consumer
 sen3ment
 and

 predic3ng
 behavior
 
Ø detec3ng
 adverse
 events
 and
 predic3ng
 their
 impact
 
Ø intelligent
 ques3on
 answering
 such
 as
 in
 Watson
 
Ø categorizing
 and
 recognizing
 places,
 faces,
 people,
 ..
 
Ø personalized
 genomic
 medicine

 of
 the
 future
 
Ø building
 more
 intelligent
 public
 services:
 energy,
 water
 
Ø 
 securing
 ourselves
 beCer
 ….
 


big
 data
 analy3cs
 
exploi3ng
 more
 efficient
 technology
 developed
 by

 web
 
companies
 for
 their
 web-­‐intelligence
 tasks
 

 
fusing
 social
 intelligence
 and
 business
 intelligence
 
web-­‐intelligence
 techniques
 on
 a
 mix
 of
 private
 and
 web
 data
 
Ø  sales
 and

 marke3ng
 
Ø  intelligent
 supply
 chains
 
Ø  digital,
 mobile,
 data-­‐driven
 business
 models
 and
 processes
 


 

 

“brick-­‐and-­‐mortar
 firms
 emula3ng
 web
 companies”
 


 

Web
 Intelligence
 and
 Big
 Data
 
“predict
 the
 future
 using
 AI
 and
 big
 data”
 
Search
 

Listen

Machine
 
 
Learning
 


 
 

Look

Learn

Informa3on
 
Extrac3on
 

Connect

Reasoning
 

Predict

Data
 Mining
 

Correct

Optimization

Big
 Data
 
Technology
 

Load

parallel
 programming
 
using
 map-­‐reduce