OOUG - Oracle Performance Tuning with AAS

Oracle Performance Tuning
Kyle Hailey
Kyle@delphix.com
kylehailey.com

Who is Kyle Hailey
 1990 Oracle
–
–
–
–
–

90 support
92 Ported v6
93 France
95 Benchmarking
98 ST Real World Performance

 2000 Dot.Com
 2001 Quest
 2002 Oracle OEM 10g

Success!
First successful OEM design

Who is Kyle Hailey
 1990 Oracle
–
–
–
–
–






90 support
92 Ported v6
93 France
95 Benchmarking

2000 Dot.Com
2001 Quest
2002 Oracle OEM 10g
2005 Embarcadero
– DB Optimizer

Who is Kyle Hailey
•

1990 Oracle






•
•
•
•
•

90 support
92 Ported v6
93 France
95 Benchmarking

2000 Dot.Com
2001 Quest
2002 Oracle OEM 10g
2005 Embarcadero
 DB Optimizer
Delphix

When not being a Geek
- Have a little 4 year old boy who takes up all my time

Production
Instance

Database

File system
File system

QA

UAT

Instance

Instance

Instance

Database

Database

Database

File system
File system

File system
File system
File system

File system
File system

Development

Production
Instance

Database

File system

Development

QA

UAT

Instance

Instance

Instance

Database

Database

Database

My Goal
Simplify the information
and empower the DBA

How do you tune a Database?
Database is running slow!
• first step?
• questions?
• tools?
• repeatable method?

What statistics do you look
at?

Disastrously difficult

Launch: Pressure

Midnight
January 28, the Flight
1986
Night before
Lives are on the line

Thanks to Edward Tufte

Estimated launch
temperature 29º

13 Pages Faxed

3 different types of names

13 Pages Faxed

Damage (in overwhelming detail) but No Temperatures

13 Pages Faxed

Temperatures but limited Damage

Test engines, not launches,
fired horizontally
Missing temperatures for 5 erosion damage flights

13 Pages Faxed

“blow by”, not more important “erosion”, (at hottest and coldest launches)

13 Pages Faxed

Recommendation

Predict
Temperature

Original Engineering data
damages at
the hottest
and coldest
temperature
- management

3
2
1

55

60

65

70

Would you launch?

75

80

Congressional Hearings
Evidence

No Damage Legend
Damage hard to read

Congressional Hearings
Evidence

Temperature
correlation difficult

Original Data

3
2
1

55

60

65

70

75

80

Clearer
12

8

4

55

60

65

70

75

80

1. Y-Axis amount of damage (not number of damage)

Clearer

12

8

4

55

60

65

70

75

80

2. Include successes

Clearer
12

8

4

55

60

65

70

75

80

3. Mark Differences

Clearer
12

8

4

55

60

65

70

75

80

3. Mark Differences
4. Normalize same temp

Clearer
Damage on every flight
below 65
12
8

4

55

60

65

70

75

80

3. Mark Differences

No damage on every
flight above 75

Clearer
12

12

8

8

4

X

4

30

35

40

45

50

55

60

65

70

75

80

3. Mark Differences
5. Scale known vs unknown

Difficult
 NASA Engineers Fail
 Congressional Investigators Fail
 Data Visualization is Difficult
But …
Lack of Clarity can be devastating

Solutions

• Clear Identification
•

Know how to identify problems and issues

• Access to details
•

Provide solutions and/or information to address the issues

• Graphics
•

Easy understanding, effective communication and discussion

First Step: Graphics
“The humans … are exceptionally good
at parsing visual information,motion
especially when that information is
coded by color and/or
.”
Knowledge representation in cognitive science. Westbury, C. & Wilensky, U. (1998)

Why Use Graphics
You can't imagine how many times I was told that nobody wanted or
would use graphics …
-- Jef Raskin, the father of the Macintosh

Infocus – (overhead projectors) sited a
study that humans can parse
graphical information 400,000 times
faster than textual data

Counties in US
 3101 Counties
 50 pages

I
y

x

Average
Standard Deviation
Linear Regression

x

10
8
13
9
11
14
6
4
12
7
5
9

8.04
6.95
7.58
8.81
8.33
9.96
7.24
4.26
10.84
4.82
5.68
7.5

3.31
1.33

2.03

Anscombe's Quartet
II
III
y
x
y
x
10
9.14
10
7.46
8
8.14
8
6.77
13
8.74
13 12.74
9
8.77
9
7.11
11
9.26
11
7.81
14
8.1
14
8.84
6
6.13
6
6.08
4
3.1
4
5.39
12
9.13
12
8.15
7
7.26
7
6.42
5
4.74
5
5.73
9
7.5
9
7.5
3.31
1.33

2.03

3.31
1.33

2.03

IV
y
8
8
8
8
8
8
8
19
8
8
8
9
3.31
1.33

6.58
5.76
7.71
8.84
8.47
7.04
5.25
12.5
5.56
7.91
6.89
7.5
2.03

“If I can't picture it, I can't understand it”
- Albert Einstein

Graphics for Anscombe’s Quartet

Do You Want?
Engineering Data?

Do You Want?
Clean and Clear

?
?

?
?

?

?

Tuning the Database

Complex
What is a Averages life look
day in the
like for a DBA who has
performance issues?
I
y

x
Average
Standard Deviation

Linear Regression

9
3.31
1.33

x
7.5
2.03

Anscombe's Quartet
II
III
y
x
y
x
9
7.5
9
7.5
3.31
2.03
3.31
2.03
1.33
1.33

IV
y
9
3.31
1.33

7.5
2.03

What statistics do you
look at?

AAS

Imagine Trying to Drive your
Car
Would you want your dashboard to look like :

And is updated once and hour

Or would you like it to
look …

How Can We Open the Black Box?

LOAD
Max CPU
(yard stick)
Top Activity

SQL

Sessions

How Can We Open the Black Box?
OEM

ASHMON/SASH

DB Optimizer

• Powerful - Identifies issues quickly and indepth
• Interactive - Allows exploring the data
• Easy - Understandable by everyone, DBA, Dev and Managers

!

Ideas for Today

AAS
ASH
Sampling
Wait
s

Average Active Sessions
(AAS)
The Most Important Metric

Kyle Hailey
Kyle@delphix.com
http://kylehailey.com

In this Session
1. AAS
–
–

Single Metric
DB Performance

1. Yardstick
–
–

CPU Count
Compare to AAS

1. AAS components
–
–
–
10/18/13

CPU
Waits
Time series

47

Database Performance
How quick can you find
Bottleneck in DB
If DB is idle
Current DB Load
what is DB Load ?

What do you use?
Statspack/AWR
V$active_session_history
Alerts

what the *!####!
*!*? is the
database
doing ?!

what do you alert on ?

10/18/13

48

Statspack: Cheat Sheet
• Install

– Connect as SYSDBA
– @?/rdbms/admin/spcreate.sql
• Run

– Exec statspack.snap;
• Generate Reports

– @?/rdbms/admin/spreport.sql
– (AWR : @?/rdbms/admin/awrrpt.sql )

Statspack = always available
Copyright 2006 Kyle Hailey

Statspack Sections (10g)

Instance description
Host hardware
Snapshot headline
Cache information
Load profile
Instance efficiency
Shared pool stats
Top timed events
Host CPU load
Host / Instance CPU warning
VM activity
Memory usage
Time model stats
RAC statistics
All wait events
Background wait events
Event histogram
SQL x 9
Instance activity
Log switches

Session details x 3
Tablespace I/O
File I/O
File I/O histogram
Buffer pool
Instance Recovery
Buffer pool advisory
Buffer busy waits
Various PGA
summaries
PGA histogram
PGA advisory
PGA allocation
summary
PGA allocation top
N
Enqueue (lock)
activity
Undo stats x 2
Latch activity
Latch miss details

Latch parent and child
Mutex
Segment stats x 8
Dictionary cache
Library cache
RAC (GES)
RAC (CR and CUR
served)
RAC (cache xfer x 2)
RAC (Remastering)
Streams x 7
Shared pool advisor
Java pool advisor
SGA resizing
SGA target advisor
SGA summary
SGA detail
SQL memory summary
Resource limits
Parameters

Statspack Method
Who/When

Summary

Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% for
Load profile goodTotal
Event
Waits having a (s) Call Time
Time feel for the
---------------------- ------------ application and
----------- --------buffer busy waits
2,748
250
78.72
comparing two 10.16
CPU time
32
periods for changes
free buffer waits
1,588
15
4.63
Efficiency 8
ratios 2.51
write complete waits
10
log buffer space
306 misleading carry 1.51
5
---------------------------------------- 6
over from version

days

Waits

Big Picture

Statspack Method

v$system_event - wait events
v$sysstat – cpu stats
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% Total
Event
Waits
Time (s) Call Time
---------------------- ------------ ----------- --------buffer busy waits
2,748
250
78.72
CPU time
32
10.16
free buffer waits
1,588
15
4.63
10
8
2.51
log buffer space
306
5
1.51
----------------------------------------

Old method:

compare wait time to CPU time

Current Method
•
•
•

1000 lines of data / 30 pages
What do you look at ?
Top 5 Timed Events?

188 minutes CPU
Elapsed 15 minutes
48 Processors or
74% idle

Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% Total
Event
Waits
Time (s) Ela Time
------------------------ ------------ ----------- -------CPU time
11,330
81.95
log file sync
163,725
1,012
7.32
enqueue
4,486
615
4.45
latch free
28,786
236
1.71
db file sequential read
1,420,355
149
1.08
----------------------------------------Jonathan Lewis - http://www.jlcomp.demon.co.uk/statspack_01.html

Analysis requires
•elapsed time
•CPU core count

Average Active Sessions (AAS)
DB Time = wait time + CPU time

AAS = DB Time/Elapsed Time

AAS like run queue on UNIX

AWR Report
• AAS = DB Time/Elapsed Time

• 23.56/59.66 = 0.39

AAS= 0.39
10/18/13

57

ASHRPT
ASH Report For TESTDB/testdb
ASH Report For TESTDB/testdb
DB Name
DB Id
Instance
DB Name
DB Id
Instance
TESTDB
TESTDB
CPUs
CPUs

Inst Num Release
Inst Num Release

2371570538 testdb
2371570538 testdb
SGA Size
Buffer Cache
SGA Size
Buffer Cache

RAC Host
RAC Host

11 10.2.0.1.0 NO sdbe604a
10.2.0.1.0 NO sdbe604a
Shared Pool
ASH Buffer Size
Shared Pool
ASH Buffer Size

22

1,000M (100%)
468M (46.8%)
112M (11.2%)
4.0M (0.4%)
1,000M (100%)
468M (46.8%)
112M (11.2%)
4.0M (0.4%)
Analysis Begin Time:
21-Apr-06 12:00:01
Analysis Begin Time:
21-Apr-06 12:00:01
Analysis End Time:
21-Apr-06 12:05:01
Analysis End Time:
21-Apr-06 12:05:01
Elapsed Time:
5.0 (mins)
Elapsed Time:
5.0 (mins)
Sample Count:
3,716
Sample Count:
3,716
Average Active Sessions:
12.39
Average Active Sessions:
12.39
Avg. Active Session per CPU:
6.19
Avg. Active Session per CPU:
6.19
Report Target:
None specified
Report Target:
None specified
Top User Events
DB/Inst: TESTDB/testdb (Apr 21 12:00 to 12:05)
Top User Events
DB/Inst: TESTDB/testdb (Apr 21 12:00 to 12:05)
Avg Active
Avg Active
Event
Event Class
%% Activity
Event
Event Class
Activity Sessions
Sessions
CPU ++ Wait for CPU
CPU
Wait for CPU
enq: TX -- row lock contention
enq: TX
row lock contention
buffer busy waits
buffer busy waits
latch: cache buffers chains
latch: cache buffers chains

CPU
CPU
Application
Application
Concurrency
Concurrency
Concurrency
Concurrency

67.98
67.98
23.98
23.98
4.66
4.66
2.26
2.26

8.42
8.42
2.97
2.97
0.58
0.58
0.28
0.28

Statspack AAS

Look for
Elapsed Time
Top 5 Timed Events
 Start at line 52 of about 1300

10/18/13

59

Statspack AAS
• Elapsed Time
STATSPACK report for
STATSPACK report for
LABSF03 1420044432
1 10.1.0.2.0 NO labsfr
LABSF03 1420044432 labsf03
labsf03
1 10.1.0.2.0 NO labsfr
Snap Id
Snap Time
Sessions Curs/Sess
Snap Id
Snap Time
Sessions Curs/Sess
Begin Snap:
1 03-Apr-06 12:34:06
18
5.6
Begin Snap:
1 03-Apr-06 12:34:06
18
5.6
End Snap:
2 03-Apr-06 12:34:36
18
4.8
End Snap:
2 03-Apr-06 12:34:36
18
4.8
Elapsed:
1.00 (mins)
Elapsed:
1.00 (mins)

• Look at Top 5 Timed Events
Top 5 Timed Events
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~
Event
Event
buffer busy waits
buffer busy waits
CPU time
CPU time
free buffer waits
free buffer waits
log buffer space
log buffer space
10/18/13

Waits
Waits Time (s)
Time (s)
2,748
250
2,748
250
32
32
1,588
15
1,588
15
10
8
10
8
306
5
306
5

% Total
% Total
Call Time
Call Time
78.72
78.72
10.16
10.16
4.63
4.63
2.51
2.51
1.51
1.51
60

Statspack AAS
• DBTIME= CPU + WAITS
– CPU = 32

– WAITS = 250+15+8+5 = 278 secs
DBTIME=320

• Elapsed Time = 60 secs
•

320 secs / 60 secs

Top 5 Timed Events
Top 5 Timed Events
Event
Time (s)
Event
Time (s)
--------------------------------- --------buffer busy waits
250
buffer busy waits
250
CPU time
32
CPU time
32
free buffer waits
15
free buffer waits
15
8
8
log buffer space
5
log buffer space
5

AAS = 5.1
CPU core count = 2
61

AAS Formulas
Use CPU count as yardstick:
 AAS < 1
Database is not blocked

 AAS ~= 0
Database basically idle
Problems are in the APP not DB

Ideal world –
one database
solution
track CPU at OS

 AAS < # of CPUs
CPU available
Are any single sessions 100% active?

•

AAS > # of CPUs
Could have performance problems

 AAS >> # of CPUS
There is a bottleneck

AAS > 1
still want to know
if a single user is
100% active

What’s the DB Doing?!

It’s 2am … your manager calls
Whip out the stethoscope:
AAS
what the *!####!*!
*? is the database
doing ?!
10/18/13


64

AAS Components
Performance Page

Real CPU available:
Max CPU - non instance CPU

Available CPU

AAS:
CPU + WAIT

OEM

Relax

Get to Work!

Looks
OK
But …

What if AAS
indicates
a problem?

Latch Free
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% Total
Event
Waits
Time (s) Ela Time
-------------------------------------------- ------------ ----------- -------latch free
9,652
760
66.10
CPU time
248
21.62
PL/SQL lock timer
41
123
10.72
SQL*Net message from dblink
681
14
1.22
log file parallel write
128
1
.13
-------------------------------------------------------------

What Latch? There are 100s

Latch Sleep breakdown for DB: CDB Instance: cdb Snaps: 3 -4
-> ordered by misses desc
Latch Name
Requests
Misses
Sleeps
-------------------------- -------------- ----------- ----------cache buffers chains
8,448,787
649,484
6,930
library cache pin
8,405,896
82,915
1,427
library cache
shared pool

8,435,488

55,645

58,626

7

Sleeps 1->4
-----------0/0/0/0/0
81537/1330/4
7/1/0
1,294 54375/1247/2
2/1/0
1 6/1/0/0/0

Row Locks 10g+
op 5 Timed Events
Avg %Total
~~~~~~~~~~~~~~~~~~
wait
Call
Event
Waits
Time (s)
(ms)
Time
----------------------------------------- ------------ ----------- ------ -----enq: TX - row lock contention
59
160
2714
41.8
PL/SQL lock timer
4
117 29291
30.6
CPU time
28
7.2
buffer busy waits
1,217
18
15
4.7
log file parallel write
422
11
27
3.0

Who is waiting
Who is blocking
What is the SQL
What is the row?

Buffer Busy Wait

Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% Total
Event
Waits
Time (s) Call Time
-------------------------------------------- ------------ ----------- --------buffer busy waits
2,748
250
78.72
CPU time
32
10.16
free buffer waits
1,588
15
4.63
10
8
2.51
log buffer space
306
5
1.51
------------------------------------------------------------

Buffer Busy Wait – trying to modify a block
Who blocks? What object? what is the SQL?
Buffer wait Statistics DB/Inst: DB4/db4 Snaps: 2-3
Class
Waits Wait Time (s) Avg Time (ms)
------------------ ----- ------------- ------------file header block
264
203
769
data block
6,070
162
27
undo header
355
0
1

Statspack fails for analysis

OEM Per formance Page: Buf fer Busy Waits

Dilema: AAS indicates problem but not
solutions
Problem in
•Statspack
•AWR
•OEM performance page
Can’t analyze
•Locks
•Buffer Busy Waits
•Latches
Also can’t see when demand for CPU is greater than
available CPU

Example
•

Look at Statspack:

Top 5 Timed Events
~~~~~~~~~~~~~~~~~~
% Total
Event
Waits
Time (s) Call Time
--------------------------- ----------- --------buffer busy waits
2,748
250
78.72
CPU time
32
10.16
free buffer waits
1,588
15
4.63
10
8
2.51
log buffer space
306
5
1.51
-------------------------------

What do we do?
•
•

How do we solve “buffer busy wait”
Need:
•
•
•
•

SQL
Sessions
Objects
Wait details

– Type of buffer busy wait
– File and block numbers
•

How do we get that information?
– Not from Statspack or AWR
– But from v$session
• v$session_wait prior to 10g

V$session
•

Shows who is waiting
– For example lists who is waiting on “buffer busy waits”
– Data only exists while waits are happening
– Data includes
• Type of buffer busy wait
• File and block involved in buffer busy wait

•
•

Problem: Once waits are over, data is gone
Solution: Sample data all the time

Sample Query
select
nvl(s.username,s.program) username,
s.sid sid,
s.serial# serial,
s.sql_hash_value sql_hash_value, SQL identifier
substr(decode(w.wait_time,
Waiting or on CPU?
0, w.event,
'ON CPU'),1,15) event ,
w.p1 p1,
w.p2 p2,
w.p3 p3
from
v$session
s,
v$session_wait w
where w.sid=s.sid
and s.status='ACTIVE'
and s.type='USER';

This query works since v7

Sampling Output
USERNAME
---------SYS
SYS
STARGUS
(CJQ0)

SID
----64
58
71
9

SERIAL
-------8717
19467
6251
1

SQL_HASH_V
---------4116021597
961168820
1311875676
0

EVENT
-------------------PL/SQL lock timer
ON CPU
direct path write
rdbms ipc message

Run sample query every second and
save into a table “v$ash”
Create table v$ash as select … ;
Insert into v$ash select … ;

P1
-------300
16508152
201
500

P2
-------0
1
2155902
0

P3
---0
0
127
0

Buffer busy wait type
SQL> Select count(*), p3
from v$ash
where event = 'buffer busy waits'
group by p3;
COUNT(*)
P3
---------- ---3423
1

P3 = block type

Buffer busy wait type
N CLASS

Block types come from
select rownum n,
class
from v$waitstat;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

data block
sort block
save undo block
segment header
save undo header
free list
extent map
1st level bmb
2nd level bmb
3rd level bmb
bitmap block
bitmap index block
file header block
unused
system undo header
system undo block
undo header

File and Block #s
select count(*), p1 filen, p2 blockn
from v$ash
where event='buffer busy waits'
group by p1, p2, hash_value;
COUNT(*)
FILEN
BLOCKN
---------- -------- -------1
11
90644
2
11
90651
3
11
98233
1
11
104767
3
11
113291
1
11
119842
1
11
119856
3
11
121632
1
11
126334

Pick a File and Block to find object

Find Object
column segment_name format a30
select

COUNT(*)
FILEN
BLOCKN
---------- -------- -------1
11
126334

owner,
segment_name,
segment_type,
block_id, blocks+block_id
from dba_extents
where file_id = 11
and 126334 between block_id AND block_id + blocks-1;
OWNER
SEGMENT_NAME
SEGMENT_TY
BLOCK_ID BLOCKS+BLOCK_ID
---------- ------------------ ---------- ---------- --------------SYSTEM
TOTO1
TABLE
125201
127249

Data block ( not header block )

126334

What SQL ?
SQL> select count(*), sql_hash_value
2 from v$ash
3 where event='buffer busy waits'
4 group by hash_value;
COUNT(*)
---------3423

SQL_HASH_VALUE
-------------558666863

All the same SQL
SQL_HASH_VALUE= 558666863
In version 10g+ use SQL_ID

SQL Statement ?
select sql_text
from v$sqltext
where hash_value=558666863;
SQL_TEXT
----------------------------------------------------INSERT into toto1 values (:b1,lpad('a',1000,'a'))

• Insert statement
• Problem on a data block on
• table toto1

Solution: Freelists or ASSM

Sampling Summar y
• Sampling v$session
– finds the root of problems
– otherwise impossible to diagnose

• Sample Data from v$session is called
– Active Session History (ASH)
– v$active_session_history
Diagnosis is still complex
But
done automatically by ADDM

OEM & ADDM Example

Four Tuning Paths
Database Home Page
Performance Page
Drilldowns
SQL

Session

ADDM Page
Solutions

Path 1
Database Home Page
Performance Page
Drilldowns
SQL

Session

ADDM Page
Solutions
Actions

ADDM Details

Tablespace

Solution ASSM
Object

Solution 2: Partitioning

Solution 3: Free Lists

Path 2
Database Home Page
Performance Page
Drilldowns
SQL

Session

ADDM Page
Solutions
Actions

EM Product Layout for Performance
Database Home Page
Performance Page
Drilldowns
SQL

Session


ADDM Page
Solutions

Database Performance Page

Lots of waits

ADDM Report

EM Product Layout for Performance
Database Home Page
Performance Page
Drilldowns
SQL

Session

ADDM Page
Solutions

Database Performance Page

“click on
the big
stuff”

Session : Wait History


Part II – Taking ADDM Advice
•

10 users inserting concurrently into same table

insert into toto1 values (t,lpad('a',1000,'a'));

ADDM Details

• Use ASSM
• Use Freelists
• Partition Object

=> Trying out Freelists

Test 2

Elapsed Time 14:59
Was 14:44 – now slower!

ADDM Details

• Increase Log File Size to 2000M to hold at least 20 minutes of redo

Test 3
14:44

Elapsed Time 1:23
10x Improvement!

14:59

Elapsed 1:23

Final ADDM Details

Reduce the number of commits
=> Commit every 50 records instead of every record

Final
•
•

Commit every 50 records instead of every record
Got elapsed down from
– 1:23
– 0:47

In Review
•

By using
– Free Lists
– Large Redo Log files

•

Job Processing went from
– Over 14 minutes to
– Just over 1 minute

•

Thanks to ADDM and OEM !

Measuring Active
Sessions

Sampling Every Second
...
User 1
User 2
User 3

Time
Time
Fast query run rarely
Query run often

Slow query

Active Session

idle

Query 1 idle

Query 2 idle

Time
Time

Query 3

idle

Active Session
idle

Query 1 idle

Query 2 idle

Time
Time
CPU

IO

Wait

Query 3

idle

Active Session

CPU

IO

Work

Wait

Latency
Contention

v$active_session_history
When

Session

State

Wait

SQL

SAMPLE_ID
SAMPLE_TIME

NUMBER
TIMESTAMP(3)

SESSION_ID
SESSION_SERIAL#
USER_ID
SERVICE_HASH
SESSION_TYPE
PROGRAM
MODULE
ACTION
CLIENT_ID

NUMBER
NUMBER
NUMBER
NUMBER
VARCHAR2(10)
VARCHAR2(64)
VARCHAR2(48)
VARCHAR2(32)
VARCHAR2(64)

SESSION_STATE
WAIT_TIME
EVENT
EVENT_ID
EVENT#
SEQ#
P1
P2
P3
WAIT_TIME
TIME_WAITED
CURRENT_OBJ#
CURRENT_FILE#
CURRENT_BLOCK#
SQL_ID
SQL_CHILD_NUMBER
SQL_PLAN_HASH_VALUE
SQL_OPCODE
QC_SESSION_ID
QC_INSTANCE_ID

VARCHAR2(7)
NUMBER
VARCHAR2(64)
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER0
VARCHAR2(13)
NUMBER
NUMBER
NUMBER
NUMBER
NUMBER

DurationTIME_WAITED

NUMBER

Primary Fields of ASH
Activity : Who, What, When, How

When

SAMPLE_TIME

Who

SESSION_ID

Session

SESSION_STATE

State

EVENT

Wait

SQL_ID

SQL

Time

How

What

Sampling is like taking Pictures

Graphical ASH

Session 1
Session 2
Session 3
Session 4

TIME

The Power

DB Home

ASH gives
AAS

Performance

Based on TIME

AAS=db
time/elapsed time

v$system_event
v$sysstat

Top Activity
AAS=count active
users /samples

Based on
ASH

2. OEM 10g : Top Activity
•Top Activity
•Based on ASH
•Enables Drilldowns
•Top SQL
•Top Session

•Drill into a session
• Stats
• Raw waits
• Open cursors
• General info

•Drill into a SQL
• Stats and text
• Users executing
• Explain plan
• Tuning options

Top Activity : Based on ASH

Thanks
To
ASH

Statistic Lag Time
ASH
Real Time

Wait Stats

Slight lag

OEM Perf Pages
Top Activity

Performance

SQL
Session

Session
SQL

AAS from ASH
1.

ASHRPT
–

Based entirely on v$active_session_history

–
–

@?/rdbms/admin/ashrpt.sql
Exec ASH_REPORT_TEXT/HTML
select * from table
(dbms_workload_repository.ash_report_text(
(select dbid from v$database),
1,
sysdate – 1/24,
sysdate )) ;

10/18/13

137

ASH RPT
1) General info
1) General info
2) Top User Events ***
2) Top User Events ***
3) Top Background Events
3) Top Background Events
4) Top Event P1/P2/P3 Values
4) Top Event P1/P2/P3 Values
5) Top Service/Module
5) Top Service/Module
6) Top Client IDs
6) Top Client IDs
7) Top SQL Command Types
7) Top SQL Command Types
8) Top SQL Statements ***
8) Top SQL Statements ***

9) Top SQL using literals
9) Top SQL using literals
10) Top Sessions ***
10) Top Sessions ***
11) Top Blocking Sessions
11) Top Blocking Sessions
12) Top Sessions running PQs
12) Top Sessions running PQs
13) Top DB Objects
13) Top DB Objects
14) Top DB Files
14) Top DB Files
15) Top Latches
15) Top Latches
16) Activity Over Time ***
16) Activity Over Time ***

ASHRPT over Time
• Waits over Time
– Not in AAS

• Difficult but better
than nothing

Compare to …

Custom Scripts
• Hate Graphics ?
– Query v$active_session_history directly
– Join to dba_hist_active_sess_history for week of data

• act.sql
– Like top 5 timed events

• Aveact.sql
– Charts with text AAS by hour (15 minute, minute , etc)

• Aveactn.sql
– Ditto, with top 2 wait events per bucket
• Following Scripts Available on

http://ashmasters.com
10/18/13

141

Custom Scripts
@act
@act
Analysis Begin Time :
Analysis Begin Time : 2007-07-24 11:04:48
2007-07-24 11:04:48
Analysis End
Analysis End Time :
Time : 2007-07-24 11:19:45
2007-07-24 11:19:45
Start time, mins ago:
15
Start time, mins ago:
15
Request Duration
:
15
Request Duration
:
15
Collections
:
528
Collections
:
528
Data Values
:
3327
Data Values
:
3327
Elapsed Time: 15 mins
Elapsed Time: 15 mins
WAIT_EVENT
WAIT_EVENT
latch free
latch free
log buffer space
log buffer space
buffer busy waits
buffer busy waits
db file scattered read
db file scattered read
library cache pin
library cache pin
log file sync
log file sync
ON CPU
ON CPU
enqueue
enqueue
sum
sum

CNT
CNT
10
10
13
13
14
14
15
15
78
78
213
213
726
726
855
855
1399
1399

% Active Ave_Act_Sess
% Active Ave_Act_Sess
.3
.02
.3
.02
.39
.02
.39
.02
.42
.03
.42
.03
.45
.03
.45
.03
2.34
.15
2.34
.15
6.40
.40
6.40
.40
21.82
1.38
21.82
1.38
25.70
1.62
25.70
1.62
42.05
2.65
42.05
2.65
6.30
6.30

Custom Scripts
@aveact
@aveact
TM
NPTS AVEACT GRAPH
CPU WAITS
TM
NPTS AVEACT GRAPH
CPU WAITS
---------------- ------ ------- ---------------------- ---- -------------------- ------ ------- ---------------------- ---- ----06-AUG 13:00:00
270
.33 +2
29
59
06-AUG 13:00:00
270
.33 +2
29
59
06-AUG 14:00:00
1040
2.24 ++--------2--341 1984
06-AUG 14:00:00
1040
2.24 ++--------2--341 1984
06-AUG 15:00:00
623
6.67 ++++------2---------438 3718
06-AUG 15:00:00
623
6.67 ++++------2---------438 3718
06-AUG 16:00:00
1088
2.59 ++--------2---335 2486
06-AUG 16:00:00
1088
2.59 ++--------2---335 2486
06-AUG 17:00:00
1104
1.26 ++----2
349 1043
06-AUG 17:00:00
1104
1.26 ++----2
349 1043
06-AUG 18:00:00
1093
1.38 +++---2
663
06-AUG 18:00:00
1093
1.38 +++---2
663 842
842
06-AUG 19:00:00
1012
1.74 ++------- 2
373 1388
06-AUG 19:00:00
1012
1.74 ++------- 2
373 1388
06-AUG 20:00:00
1131
.99 +---2
304
06-AUG 20:00:00
1131
.99 +---2
304 820
820
06-AUG 21:00:00
1111
1.22 ++----2
344 1012
06-AUG 21:00:00
1111
1.22 ++----2
344 1012
06-AUG 22:00:00
1010
1.66 ++------ 2
414 1259
06-AUG 22:00:00
1010
1.66 ++------ 2
414 1259
06-AUG 23:00:00
1120
1.08 +---2
298
06-AUG 23:00:00
1120
1.08 +---2
298 913
913
07-AUG 00:00:00
1024
.83 +--2
273
576
07-AUG 00:00:00
1024
.83 +--2
273
576
07-AUG 01:00:00
1006
1.74 ++------- 2
319 1428
07-AUG 01:00:00
1006
1.74 ++------- 2
319 1428
07-AUG 02:00:00
1090
2.47 ++--------2---347 2345
07-AUG 02:00:00
1090
2.47 ++--------2---347 2345
07-AUG 03:00:00
687
6.59 +++-------2---------382 4142
07-AUG 03:00:00
687
6.59 +++-------2---------382 4142
07-AUG 04:00:00
1004
1.95 ++++++--- 2
1299
07-AUG 04:00:00
1004
1.95 ++++++--- 2
1299 659
659
07-AUG 05:00:00
1104
3.08 +++++-----2-----1170 2226
07-AUG 05:00:00
1104
3.08 +++++-----2-----1170 2226
07-AUG 06:00:00
1122
1.91 +++++++-- 2
1582
07-AUG 06:00:00
1122
1.91 +++++++-- 2
1582 558
558

Aveact.sql

“-” = WAIT
“+” = CPU
which waits ? -> aveactn.sql

Before Tuning Oracle, Tune the Machine

Make sure the machine is healthy before tuning Oracle
 CPU => use run queue, < 2 * #CPU
 Memory => page out
VMSTAT

Summary

1.Machine - vmstat
–

Memory, CPU (we can see IO response in Oracle)

1.Database - AAS
–

Use wait interface and graphics

–

Identify machine, application, database or SQL

1.SQL - VST
–

Indexes, stats, execution path

–

Visual SQL Tuning

Bibliography

Refactoring SQL Applications - Stephane Faroult
Troubleshooting Oracle Performance - Christian Antognini
SQL Tuning - Dan Tow
Oracle Wait Interface - Kirtikumar Deshpande, K.
Gopalakrishnan

Books
The Visual Display of

Design

Quantitative Information
– Eduard Tufte

For the Space Shuttle disaster, see
http://williamwolff.org/wp-content/uploads/2013/01/tufte-challenger-1997.pdf
http://www.onlineethics.org/cms/12709.aspx

Using graphics for
database

System Tuning

SQL Tuning

152

ASSM ?
select FREELISTS, TABLESPACE_NAME
from dba_tables
where table_name='TOTO1' and owner='SCOTT';
FREELISTS
---------1

TABLESPACE_NAME
-----------------------------USERS

Table only has 1 freelist
The table is in tablespace Users
Is it an ASSM tablespace ?

Sampling Output
select tablespace_name, SEGMENT_SPACE_MANAGEMENT
from dba_tablespaces;
TABLESPACE_NAME
-----------------------------SYSTEM
UNDOTBS1
TEMP
USERS
TOOLS

SEGMEN
-----MANUAL
MANUAL
MANUAL
MANUAL
AUTO

USERS tablespace is not in ASSM mode


Let’s add Freelists
How many freelists?
select count(*), sid, serial
from v$ash
where event='buffer busy waits‘
group by sid, serial;
COUNT(*)
SID SERIAL
---------- ----- -------7
12
79
4
14
99
4
19
770
8
20
176

Max Concurrency is 4
Freelists >= 4


OOUG - Oracle Performance Tuning with AAS

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a OOUG - Oracle Performance Tuning with AAS

Similar a OOUG - Oracle Performance Tuning with AAS (20)

Más de Kyle Hailey

Más de Kyle Hailey (20)

Último

Último (20)

OOUG - Oracle Performance Tuning with AAS

Notas del editor