Sep 2017 understanding business intelligence huy nguyễn

Understanding Business
Intelligence

Huy Nguyen
CTO, Cofounder - Holistics
Software


About Me
Education:
● Pho Thong Nang Khieu, Tin 04-07
● National University of Singapore (NUS), Computer Science
Work:
● Software Engineer Intern, Facebook (California, US)
● Software Engineer Intern, SenseGraphics (Stockholm,
Sweden)
● Data Infrastructure Engineer, Viki (Singapore)
Now:
● Co-founder & CTO, Holistics Software
● Co-founder, Grokking Vietnam
facebook.com/huy bit.ly/huy-linkedin


Agenda
• Architecture of BI System
• Data Thinking
• BI Implementation


Data Thinking

Holistics.io


Data Thinking
The process of articulating and formulating a dataoriented approach to measure and answer the
business question.


Ask Question
(Hypothesis)

Formulate Data
Approach

Data thinking

Draw
Insights

Get Data
(build report)

Write SQL query


Example: How often do buyers buy again?
(retail/ecommerce business)


Visualize Data → Chart

1. Chart: Visualize a chart that could give you the
answer you need
2. Table: Draw out the underlying data table that powers
the chart
3. Query: Write code that generates the table
4. Report: Use BI tool to build visualization

Date Period

Male Users

Female Users

2017-01-01

300

102

2017-01-02

260

300

...

...

select

L.listing_date,
count(1) as daily_listings,
from listings L
where L.listing_date >= NOW() - INTERVAL '30 days'
group by 1
order by 1 DESC


Data-informed vs. Data-driven
Datadeprived
Difficult to access data
Make decision by
intuition.
Usually in traditional
businesses

Data-informed
Data is only used as a
supporting tool, as
part of a bigger
context.

Data
Driven
All decisions driven by
data
Usually missing the
bigger picture

Strategic level

Works at tactical level


While data is concrete, it is
often systematically biased.
- Andrew Chen

/>

Summary: The Data Process

Ask Question
(Hypothesis)

Formulate Data
Approach

Draw
Insights

Collect Data
(add tracking)

Get Data
(build report/run
analysis)

Data Infrastructure / BI Infrastructure

Architecture of BI System

Holistics.io


Business Intelligence (n)
Business Intelligence (BI): technologies, applications
and practices for the collection, integration, analysis,
and presentation of business information


Where Your Data Are Stored
CRM/Sales/Marketing
Auto

1

2

Finance
History

User Behavior

3
Product Data

4
5

6

Advertising

Communications

/>

BI, From End User Perspective
Sales
Data
Live
Live
(ERP / CRM)

Databases
Databases

Marketing Data
(FB Ads,
Adwords, etc)

Other Data
(CSVs / Excels /
Google Sheets)

BI

Reports /
Dashboards /

Insights


Sales
Data
Live
Live
(ERP / CRM)

Databases
Databases

Other Data
(CSVs / Excels /
Google Sheets)

Reports /
Dashboards /
Insights

BI

Marketing Data
(FB Ads,
Adwords, etc)

Storage
Engine

Processing

Engine

Visualization
Engine


End-to-End BI Overview
External data from
CRM & Marketing
channels

Data

Information

Insights
Queries & Metrics

Standardized
Integrated
NoSQL

RDBMS
High-Available Operational
databases that run your
business

sf
Data Warehouse

Centralized
Automated
Modeled

Organized views of key data points Cleaned, Merged, Summarized, and
Tuned to perform

Tracking and knowing the key
performance indicators of your business
Dashboards & Embedded Analytics
Managed presentation of impactful
insights to internal & external decision
makers
Scheduled Emails
Regular and Event-Driven delivery of
actionable information

Custom Information
maintained by teams

Consolidate

Manage
(clean up, process)

Analyze
(explore, adhoc
analysis)

Reports /

Insights
(automated reporting)


Sales
Data
Live
Live
(ERP / CRM)

BI

Databases
Databases

Marketing Data
(FB Ads,
Adwords, etc)

Data Warehouse

Adhoc / Other
Analysis

Data Science / ML
Other Data
(CSVs / Excels /
Google Sheets)

Operational Data

Data Warehouse

Reporting /
Analysis

Holistics.io


The Data Team - Tech Company

Data Engineer

Enabler & Protector

Data Analyst

Analyzer

Data Scientist

Enhancer
“How”


Data Team: Comparing Roles
Data Engineer

Data Analyst

Data Scientist

● Background:
Programming

● Background: Business,
SQL, Excel

● Background:
statistics, maths

● Responsible: data
infrastructure

● Understand core
business,
performance metrics

● Responsible: build
recommendation
engines, predictive
models, etc

● Responsible: create
reports, look for
insights


Data Engineer

Application
Live
Live
Data

Data Analyst

Databases
Databases

Reports + Insights
Collecting Data
FB Ads,
Adwords...

Data Warehouse

Machine Learning +
Data Science
CSVs / Excels /
Google Sheets

Data Scientist

Processing Data

Operational Data

Data Warehouse

Reporting /
Analysis


What are the types of data sources?

Custom Data
Data updated and managed
in spreadsheets by
employees
Mailing Lists, Sales Call
Notes, Customer Profiles
CSV, Excel, Google Sheets,
Web Exports

Behavioral/Log Data
Track user behavior and
interactions
Pages visited, buttons
clicked, time spent,
downloads, video control
clicks
Google Tag Manager, Piwik,
Mixpanel, Localytics

Transactional Data
(OLTP)
Contains unique identifier,
numeric value, reference
data object(s)/dimension(s)

Ticket sales, Invoices, User
details, Logistics, Deliveries
SQL Databases, noSQL
databases, Web
Applications

/>

BI Implementation

Holistics.io


Implementing A BI Project?
● Approach:
○ Top-down / Strategic / Waterfall
○ Bottom-up / Tactical / Agile
● Common Problems:
○ Requirements for report always change
○ Most report requests have immediate
urgency
○ Adhoc requests are high % of the
information delivered
● Implementation Process:
○ Evaluate a BI platform
that can do the
Manage
Consolidate
following:
(clean up, process)

Analyze
(explore, adhoc
analysis)

Reports /
Dashboards
(automated reporting)


BI Hierarchy of Needs
Stage of
Company
“When”

Needs
“What”

Operations
“How”
Speed

Cost

Execution & Changes

Implementation
& Ownership

Governance

Scalability
Audit
and Processes

$$$$$

Data Volume & User Growth

r
Custome
&
Growth
ng
Increasi
s
re
ue
artk
ea
FM
t
Product
Fit

&
Product
r
Custome
Dev
Copyright © Holistics.io

Customizable Solutions

Weeks - Months

Datamarts, Data Mining & Custom Views

BI Reporting
Dashboards, Reports, and Self-Serve Querying
Hours - Days

Minutes
- Hours

$$$$$

$$$$$

People
(# and Types)