krumm04 introduction to earth science

Addressing Agency Database
Needs:
A Recent Experience From
Illinois
Donald McKay, Damon Garner, Deborah Griest,
Stephen Gustison, Donald Keefer,
Robert Krumm, Alison Lecouris, Brent Lemke,
Daniel Nelson and Barbara Stiff
Illinois State Geological Survey


Introduction
•

•
•

Recent Effort to Access Data
Management
Needs of the Illinois St. Geological
Survey
Internal Effort
Database Needs Assessment


Background Info.
•
•
•

•

•

•

Illinois Survey – about 200 people
20 years as a GIS user site (Arc license #27)
More staff using technology – GIS, database,
graphics, web to accomplish project goals
Oracle, Access, ArcGIS, Rockworks & more
Many concerns about database issues – access,
archival of data, conversion from paper to
digital,
database structure, policies – both operational
and administrative
Concerns about the agency’s ability to provide
better support for IT operations – databases,
GIS, web, more


Background Info.
•
•
•
•
•
•

•
•

Support staff stretched thin
Annual project planning cycle indicates key
staff members are in very high demand
Oracle, ArcGIS, ArcIMS, SDE, application
development, web
Many demands, expectations and desires
A recipe for disaster….?
To address concerns, a consultant to perform a
review of database operations, assess needs,
and make recommendations…?
Show me the (no) money
Decision was made to form an internal team to
conduct the review


The Process
•
•
•
•
•

•
•
•

Form an internal team – Sept. 2003
Database Management Coordinating Comm.
Chaired by Don McKay, Principal Geologist &
Chief Scientist, an Experienced GIS User

Nine team members from many program areas
Scientists from Oil & Gas, Quaternary Geology,
Groundwater, Environmental Studies and
Assessments Sections
Key support staff from Geologic Records Unit
and Financial Office
Technical staff - Oracle DB Admin, GIS DB
Admin
Representation from many sectors – experience
with database issues


Data Management Coordinating
Comm.
•
•

Initial Meetings - Sept. 2003
Identified issues including the need for easy
desktop access to database holdings,
consideration of data
at all levels – digital and paper, level of staff
support, training needs, application development
needs, providing web access to data, and more


Mission

Thrusts

Programs

Products

ISGS
3-D
Geologic
Mapping

Chief

Carbon
Management

Centers

Applied &
Education,
Basic
Operations
Tech Asst. &
Research & Support
Outreach

Data &
Information

Sections

Assets

Data Management Coordinating
Comm.
•

•

•

Talk with staff – provide an opportunity to
gather
input from all staff
‘Interviews’ with Managers and Program
Directors of all Centers, Sections and Units –
provide an opportunity to meet with staff
members,
listen to their comments and key concerns
Before the interviews, committee prepared a 3page
list of bullet points on various DB topics. The
list of topics and questions served as a
guideline


Discussion Topics -General
•

•

•

•
•

Imagine 5 years from now. What would you
like to
be able to do with your projects and your data?
What are the principal barriers to capturing,
automating or using the data needed or
generated by your section?
What are your most important data mgt.
needs?
Which of your unit's important jobs would be
most improved by better data management?
What examples of good data management in
your section might be generalized for others in
the Survey?


Discussion Topics –
Data & Information
•
•
•
•

•

What do you consider to be data?
What mission-critical data does your program
require?

What type of data does your program generate?
How much of the data that your program
generates
can be described as:
- map/spatial data; tables of data;
digital documents?
- other digital images; hardcopy documents?
- physical collections; other?
How much of your data use involves:
- mapped views of your data?
- tabular views or summaries of data?


Discussion Topics – Operations
What input does your section do to add to your
data?
• key enter data; scan/collect digital images;
digitize vector maps; scan documents?
• create reports; generate analyses;
make interpretations?
• import datasets; other, please specify.
What are the ultimate outputs of your work
flow?
• Images; maps; documents; reports;
• analyses; interpretations
• hard copy products; CD products
• web-based presentations; web-based data
delivery

Discussion Topics – Operations
What are the essential analytical or processing
functions and "objects" that occur within your
data
stream/work flow, between inputs & outputs?
• Tabulation/reporting; statistical analysis;
visualization; modeling; other?
• Who in your section is involved in DB mgt.
activities?
If responsibility for coordinating import,
organization,
and management of your unit's data were
centralized
with one or more of your staff:
• What barriers do you see to success?
• What advantages do you see?


Discussion Topics –
Users and Customers
Who are the known and potential users of your data in
the following categories?
• Your section/unit; Survey; External
Please estimate the percentage of data access
needs related to your programs that service:
• Your section/unit; Survey; external
Of the consumers of your data/information:
• Who are the most important?
• Who are the most frequent?
Please characterize your staff’s database management

experience by number of individuals operating at:
• beginning level; intermediate level; advanced level


Conducting the
Needs Assessment
•
•
•

•

•

Database Team held 10 meetings (20
hrs.)
Met with 50 people
Received input from every program area
within the agency – scientific staff &
support staff
Received input from a broad crosssection of
people, in terms of DB/IT experience
Input from meetings, as well as written
input – based on key issues (Data &
Information,


Notable quotes
•
•

•
•
•

•
•
•

•

“I don’t know how to use existing tools or
databases.”
“We are storing gigabytes of data on unstable
media.”
“I can’t get help to do what needs to be done.”
“Our lab equipment is generating digital output,
that is not being captured for lack of DB help.”
“Our data are misplaced, discarded, and orphaned,
daily.”
“Our important applications are not maintained.”
“I can’t access important data, maps & records
easily.”
“Huge amounts of our data on paper have no
prospect
of becoming digital.”
“Much of our data is completely undocumented.”


Needs Assessment - Results
•

•
•
•

After meetings with staff, draft
report
was prepared
Presented to senior management
team – March ’04
Presented to all staff – May ’04
Findings & Recommendations


Findings
We have:
• Huge data holdings
• Tradition of digital data management
• Needs outpacing support capabilities
• Complex data management
environment
• Sophisticated & underserved users
• Best available high-end software
• Sufficient hardware & network


Findings cont.
We Need:
•
•

•
•

•
•

A catalog of our data holdings
Easy access to our existing digital data
Help in automating our key data
Applications to allow us to interact with our data
A support infrastructure not understaffed and
overwhelmed
Better overall DBM to
•
•
•

•
•

preserve our results
improve our projects & products
make our jobs easier

An effective information management policy
In short, we need “HELP!”


DBMCC’s Recommendations
1.

2.
3.
4.
5.
6.
7.

Adopt a Bold Information Management Strategy
Assume New and Altered Roles and Responsibilities
Develop Needed Applications
Conduct More Frequent and Relevant Training
Increase DB Staffing and Change Hiring Practices
Increase Funding for Data Management
Maintain Adequate HW/SW Infrastructure


Adopt A Radically
New Strategy
•

Focus on ISGS: serve ourselves 1st

•

Clarify & distribute DBM responsibility

•

Formalize a Database Management
Oversight Team

Designate Center/Section/Unit
Liaisons
Formalize a Core Database Staff

•
•


Strategy (cont.)
•
•

•

•
•

Collaborate on priority solutions
Hire expertise in DB Coordination &
DB Design
Provide guidelines, tools, and
techniques
Provide training
Implement coordinated distributed
database management, including
•
•
•

Individual Data

Center, Section, & Project Data
Shared Data Repository


Assume New & Altered Roles
Managers
•
•

Support improved DBM for the long haul
Provide appropriate staff & funding

Oversight Team (advisory to Chief & SMT)
•
•

Represent all Survey sectors
Recommend Survey priorities

Core DB Staff (Oracle & GIS DBAs, new hires)
•
•
•

Provide DB design & coordination
Provide guidelines & techniques
Maintain shared data repository


New & Altered Roles cont.

Liaisons
•
•
•
•

Communicate priority needs
Become expert in appropriate tools
Oversee section/project data
Help section & project staff

Everyone
•
•
•
•

Assess the status of your data
Learn to use appropriate tools
Adopt standards as they emerge
Prioritize & communicate your needs


Hiring
Hire Key Expertise
•
•
•

Database Coordinator-Programmer

Database Designer-Programmer
Applications development
- Mapping, Energy, Web, Business

•

Technical Assistants

Adjust Hiring General Practices
•

Favor qualified, database-capable
candidates


Distributed Information Management, Liaison Role,
and Coordination Environment
A database liaison
represents and provides
support for a given unit,
project or service.
Together, liaisons and
core database staff
cooperatively build the
ISGS information system.

Liaison
Liaison

Section

Liaison

Liaison

Core
Staff

Liaison

Center
Liaison

Service

Liaison

The database
management oversight
team provides a formal
mechanism for
communication, input,
and participation, and
recommends policies
and priorities to
management.

Liaison

Shared

Methods
& Goals

Shared

Liaison

Liaison
Liaison

Project
Liaison

Liaison

Service

DMOT

Liaison

Liaison
Liaison

Section

Center

Project
Section

Center

Service

Project

Core
DB

Service

Center

Project
Section

Unit- and functionspecific databases
are related yet
distinct. They are
developed as
needed and are
integrated with
core and other
databases as
appropriate.