Hanging around in my hotel in Chicago because it is raining, I have been cruising around in the OTN forums which inspired me to write something about the High Watermark and the Oracle 10gR1 New Feature SEGMENT SHRINKING.
The High Watermark is the maximum fill-grade a table has ever reached.
Above the high watermark are only empty blocks.
These blocks can be formatted or unformatted.
First let’s have a look at the question when space is allocated
- when you create a table at least one extent (contiguous blocks) is allocated to the table
- if you have specified MINEXTENTS the number of MINEXTENTS extents
will be allocated immedaitely to the table
- if you have not specified MINEXTENTS then exactely one extent
will be allocated (we will look at extent sizes later in another post).
Immediately after creation of the segment (table) the high watermark will be at the first block of the first extent as long as there are no inserts made.
When you insert rows into the table the high watermark will be bumped up step by step.
This is done by the server process which makes the inserts.
Now let us take a look at when space is released again from a segment like a table or index:
Let’s asume that we have filled a table with 100’0000 rows.
And let’s asume that we deleted 50’000 rows afterwards.
In this case the high watermark will have reached the level of 100’000 and will have stayed there. Which means that we have empty blocks below the high watermark now.
Oracle has a good reason this: it might occur that you delete rows and immediately this you insert rows into the same table. In this case it is good that the space was not released with the deletes, because it had to be get reallocate again for the following inserts, which would mean permanent changes to the data dictionary
(=> dba_free_space, dba_extents, dba_segements …) .
Furthermore the physical addresses of the deleted row get recycled by new rows.
These empty blocks below the high watermark can get annoying in a number of situations because they are not used by DIRECT LOADs and DIRECT PATH LOADs:
1. seriell direct load:
INSERT /*+ APPEND */
INTO hr.employees
NOLOGGING
SELECT *
FROM oe.emps;
2. parallel direct load:
ALTER SESSION ENABLE PARALLEL DML;
INSERT /*+PARALLLEL(hr.employees,2)
INTO hr.employees
NOLOGGING
SELECT *
FROM oe.emps;
3. direct path loads:
sqlldr hr/hr control=lcaselutz.ctl … direct=y (default is direct=n)
All the above actions case that the SGA is not used for the inserts but the PGA:
there wil be temporary segements filled and dumped into newly formatted blocks above the high watermark.
So we might want to get high watermark down before we load data into the table in order to use the free empty blocks for the loading.
So how can we release unused space from a table?
There are a number of possible options which are already available before Oracle 10g:
- What we always could do is export and import the segment.
After an import the table will have only one extent.
The rows will have new physical addresses and
the high watermark will be adjusted.
- Another option would be to TRUNCATE the table.
With this we would loose all rows which are in the table.
So we cannot use this if we want to keep existing records.
With Oracle 9i another possibilty was implemented:
ALTER TABLE emp MOVE TABLESPACE users;
This statement will also cause that
- the rows will have new physical addresses and
- the high watermark will be adjusted.
But for this:
- we need a full (exclusive) table lock
- the indexes will be left with the status unusable (because they contain the old rowids) and must be rebuilt.
Starting with ORACLE 10gR1 we can use a new feature for adjusting the high watermark,
it is called segment shrinking and is only possible for segments which use ASSM, in other words, which are located in tablespaces which use Automatic Segement Space Management.
In such a tablespace a table does not really have a High watermark!
It uses two watermarks instead:
- the High High Watermark referred to as HHWM, above which alle blocks ar unformatted.
- the Low High Watermark referred to as LHWM below which all blocks are formatted.
We now can have unformatted blocks in the middle of a segment!
ASSM was introduced in Oracle 9iR2 and it was made the default for tablespaces in Oracle 10gR2.
With the table shrinking feature we can get Oracle
to move rows which are located in the middle or at the end of a segment
further more down to the beginning of the segment and by
this make the segment more compact.
For this we must first allow ORACLE to change the ROWIDs of these rows by issuing
ALTER TABLE emp ENABLE ROW MOVEMENT;
ROWIDs are normally assigned to a row for the life time of the row at insert time.
After we have given Oracle the permission to change the ROWIDs
we can now issue a shrink statement.
ALTER TABLE emp SHRINK SPACE;
This statement will procede in two steps:
- The first step makes the segment compact
by moving rows further down to free blocks at the beginning of the segment.
- The second step adjusts the high watermark. For this Oracle needs an exclusive table lock,
but for a very short moment only.
Table shrinking…
- will adjust the high watermark
- can be done online
- will cause only rowlocks during the operation and just a very short full table lock at the end of the operation
- indexes will be maintained and remain usable
- can be made in one go
- can be made in two steps
(this can be usefull if you cannot get a full table lock during certain hours:
you only make the first step and adjust the high watermark later
when it is more conveniant:
- ALTER TABLE emp SHRINK SPACE; – only for the emp table
- ALTER TABLE emp SHRINK SPACE CASCADE; – for all dependent objects as well
- ALTER TABLE emp SHRINK SPACE COMPACT; – only makes the first step (moves the rows)
)
How are the indexes maintained?
In the first phase Oracle scans the segment from the back to find the position of the last row.
Afterwards it scan the segment from the beginning to find the position of the first free slot in a block in this segment. In case the two positions are the same, there is nothing to shrink. In case the two positions are different Oracle deletes teh row from the back and inserts it into the free position at front of the segement. Now Oracle scan teh segement from the back and front again and again until it finds that the two positions are the same.
Since it is DML statements performed to move the rows, the indexes are maintained at the same time. Only row level locks are used for these operations in the first pase of SHRINK TABLE statement.
The following restrictions apply to table shrinking:
1.) It is only possible in tablespaces with ASSM.
2.) You cannot shrink:
- UNDO segments
- temporary segments
- clustered tables
- tables with a colmn of datatype LONG
- LOB indexes
- IOT mapping tables and IOT overflow segments
- tables with MVIEWS with ON COMMIT
- tables with MVIEWS which are based on ROWIDs
The Oracle 10g Oracle comes with a Segment Advisor utility.
The Enterprise Manager, Database Control, even has a wizzard which can search for shrink candidates.
This advisor is run automatically by an autotask job on a regular basis in the default maintainance window.
You can use the built in package DBMS_SPACE to run the advisor manually as well.
I will blog about this later on some time.
its pretty good
Explained very well….
Thank You
Beautiful! Just What I wanted.
Very good explanation. Thank you
nicely explained. even an amateur can understand
It’s what i wanted.
But how to make the index shrink more quickly?
It seems spent much time to execute the statement especially when the table is very “big” .
Thank you.
Nice explanation. Thanks
It is very good and helpful
Very good. I was trying to run this is the OEM database control, but it did not work. It does in sqlplus if I use the ENABLE ROW MOVEMENT statement first that you mentioned.
A script would be handy that showed what difference the shrinking made (one you run before and after and compare).
A very good article
its realy good one
Nice article ..thanks for sharing
Good explanation in simple words. Thanks a lot.
Very good explaination helped me lot
very nice expaination help me alot
Good Explanation. Its awesome..!!!
superb explanation . i was trying to understand from many days from diff books. but just got it in one time when reading this. thanks a lot
Thanks for the feedback aalok,
=;-)
Very nice article !! Keep it going….:)
Hi,
Excellent, beautifully placed. You have a GIFT. Very easily understood. How is it possible to be your student? How can I learn from you one to one. I think you must have my e-mail. Could you please mail me the details (if you teach that is).
Regards…..
Its really good to read this… you explained it so simply and its so easy to understand…
Nice explanation!!!
Nice Explanation, i like this article and helpful for understanding quick and easily
ITs pretty good and clean explainaton which i have used in the past for my oracle legacy applications resulted in good performance improvemnet.
Nice explanation. You ‘da man. Would you please tell me if I can do the below on a OLTP table,we cannot afford any lock on this huge tables.
ALTER TABLE TRANSACT SHRINK SPACE COMPACT;
Table TRANSACT has got about 100M+ rows and we get about 20 transactions per second. It would be nice to explain what impacts this statement going to makse such as cpu usage, redo logs consumption and UNDO.
Its superb explanation
Its Perfect. Simple and Short.
This is exactly what i was searching for.
It has helpled me a lot at the right time.
Looking forward to see may articles of troubleshooting for you.
Want to interact with you personally ,through mails atleast.
Pls do reply.
Nice Explanation….
Do you have any script to check the tables which are candidates for reorg in the application schema.
Cheers
Neeraj Rath,India
very nice explanation,
From this, I got a clear idea about the High water mark level
Thank you very much
here you gave the command for segment impact( first step: moving or rows).
then, what is the command for adjusting the water mark level only?(only for second step)
It is really a nice elaboration. At last i am bit confused whether how the index will be intact and does not require any rebuld.
If possible please clear this.
Thanks
Tapak
+919850965835
Hi Tapak,
here is my explanation of how TABLE SHRINK works:
In the first phase Oracle scans the segment from the back to find the position of the last row.
Afterwards it scan the segment from the beginning to find the position of the first free slot in a block in this segment. In case the two positions are the same, there is nothing to shrink. In case the two positions are different Oracle deletes teh row from the back and inserts it into the free position at front of the segement. Now Oracle scan teh segemtn from the back and front again and again until it finds that the two positions are the same.
Since it is DML statements performed to move the rows, the indexes are maintained at the same time. Only row level locks are used for these operations in the first pase of SHRINK TABLE statement.
I will adjust this post for this I think. Hope it helps.
=;-)
BR,
Lutz
Excellent..
Good article.
But I still looking for how to do it in parallel… I’ve a table with around 100 gb and it is taking a loooooong time to shrink.
Any idea?
Thanks,
Heitor
hello sir,
thank you sir, because that iam not able to understand till now what is high water mark & low water mark. i have lot of confusion on that. now i cleared abt high & low water mark concept. then i need segment space management on performence tuning concept. canu give me the sugestions on this concept. because performence tuning at segment level is importent. can you give me the clear my problem.
thanking you sir,
pavan
Hey,
Awesome explaination. Thank you so very much
In my case table shrink didn’t worked. After I deleted 6 table partitions on datafile had been released 150 GB.
I run table shrink but and after table shrink the High watermark is on same boundary. I still can not release free space on the datafile.
I use oracle db 10.2.0.4 with RAC option on Linux Platform.
Excellent one. Really Helped me
Hey after a long time i am seeing such a simple and clear explanation…
we had a argument with my friends on how the high water mark changes with alter table shrink command, I can refer them to this link.
do you have any queries to find out which tables are eligible for segment shrink.
Keep up your good work.
Thank You
Murali Krishna Varma . P
Good article BUT enabling row movement on table in 10G seems to invalidate all objects that use that table.
i believe its fixed in 11g
Anybody else have this issue ?
Hi,
Why does alter table shrink generates so many archives?
I thought it would generate very little redo.
regards,
Ivan
hello sir,
After I finish shrink my table, I note that all trigger belong to that table became Invalid status.
Can you explain me ?
Is there a script to find where the HW is and then create the resize statements? I want to recover space back to the OS.
SHRINK will not work with the Table who have the Functional indexes.
excellent explanation.Really I impressed a lot.I am very lucky to see your explanation and this site .
thanks&Regards,
ramji mulukoori.
nice explain ..
thanks
abhijit
Superb explanation in very easy language……..thank you
Great write up! Very clear and complete explaination. Extremely useful. Thank you for sharing.
To add one more restriction: I tried to shrink the HWM of a Hybrid Columnar Compression table in Exadata V2, it won’t work. Keep complaining “ORA-10635: Invalid segment or tablespace type”.
Gr8 explanation but just wanted to update regarding the alter table move command ,this command releases the logical space in the tablespace but we will not be gaining physical space at the O/S level due to high water mark,not sure if the shrink command helps us gain the physical space at the O/S level,will have to check the session trace and find out how it works.
Its Really Good, I tried the same and succeed., Thanks for your Clear explanation.
Rajesh
Oracle Apps DBA
WIPRO.
Very good explanation. I had a 130mil row table, 35gig – we deleted half the data and needed to shrink the space.luckily its partitioned. shrinking the whole table was blowing back the undo space. So I’m trying the alternative of shrinking each partition separately. using alter table modify partition shrink space compact.
That seems to be running ok.
shrinking the whole table was blowing back the undo space error.
any one can help me?
High Ram,
as I wrote in the post: the shrink feature uses DML (DELETE, INSERT) to compact the segment. This causes exclusive rowlocks to be created, produces UNDO data, maintains the indexes and is possible ONLINE.
If there are many rows which must be moved from the back of the segment to its front part, of course there is heave DML going on.
This happens as one atomic transaction and therefore your UNDO segments might grow dramatically.
For the second part of the shrink operation, the actual adjusting of the HWM you even need an exclusive FULL TABLE LOCK.
Hope this helps.
BR,
Lutz
We can not use this option on a compressed table. Why is it so?
Hi,
Nice article
I also found that if the table has context index, it can’t shrunk either, raises error. I’m not sure if the context index is also considered function based index. No where it mentioned shrink doesn’t work for context index.
Another question.
How does the segment shrink is different from deallocating unused space with below command?
ALTER TABLE table DEALLOCATE UNUSED
Shrink is used to shrink space below and above the HWM , whereas Deallocate Unused is used to shrink space above HWM.
Very nice information. I have the same scenario which i can test the same.
Understandable and helpful explaination
Nice explanation
VERY GOOD EXPLANATION THANKS A LOT
Really nice to article … awesome
Hey,
I think, I am tooooooooo late but never to read this wonderful article.
It was one of the best article on the web. I really appreciate the writer and appreciate him for the time he spent on this article.
Thanks,
Rakesh
Excellent! Just what i need it….
Simple, Clean and Perfect. Take a bow, Lutz.
Very much useful and very nice explanation .Thanks!