{"id":1602,"date":"2010-05-07T12:00:00","date_gmt":"2010-05-07T12:00:00","guid":{"rendered":"http:\/\/orcldoug.com\/blog\/?p=1602"},"modified":"2010-05-07T12:00:00","modified_gmt":"2010-05-07T12:00:00","slug":"statistics-on-partitioned-tables-part-6d-copy_table_stats-a-light-bulb-moment","status":"publish","type":"post","link":"http:\/\/orcldoug.com\/blog\/2010\/05\/07\/statistics-on-partitioned-tables-part-6d-copy_table_stats-a-light-bulb-moment\/","title":{"rendered":"Statistics on Partitioned Tables &#8211; Part 6d &#8211; COPY_TABLE_STATS &#8211; A Light-bulb Moment"},"content":{"rendered":"<p>I&#8217;m pretty self-concious of the amount of waffle that surrounds any technical content here, so let&#8217;s get the technical bit out of the way first, then the waffling can come later &#8230;<\/p>\n<p>I finally tracked down the mistake I <em>didn&#8217;t<\/em> make in <a href=\"http:\/\/18.133.199.212\/?p=1596\">part 6a<\/a>, but <em>thought<\/em> I&#8217;d identified and fixed in <a href=\"http:\/\/18.133.199.212\/?p=1598\">part 6b<\/a>! Here are the two sets of subpartitions for the P_20100209 partition that&#8217;s the source of the statistics and for the new P_20100210 partition that the stats were copied to. This is how <a href=\"http:\/\/18.133.199.212\/?p=1596\">part 6a<\/a> originally looked<\/p>\n<pre>TEST_TAB1                      P_20100209_GROT                NO  22-APR-2010 11:24:12          3\nTEST_TAB1                      P_20100209_HALO                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_JUNE                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_OTHERS              NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100210_GROT                NO\nTEST_TAB1                      P_20100210_HALO                NO\nTEST_TAB1                      P_20100210_JUNE                NO\nTEST_TAB1                      P_20100210_OTHERS              NO\n<\/pre>\n<p>and here it is after I&#8217;d fixed it.<\/p>\n<pre>TEST_TAB1                      P_20100209_GROT                NO  22-APR-2010 11:24:12          3\nTEST_TAB1                      P_20100209_HALO                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_JUNE                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_OTHERS              NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100210_GROT                NO  22-APR-2010 11:24:10          3\nTEST_TAB1                      P_20100210_HALO                NO                                 \nTEST_TAB1                      P_20100210_JUNE                NO                                 \nTEST_TAB1                      P_20100210_OTHERS              NO                                 \n<\/pre>\n<p>So originally I had seen no statistics on the P_20100210_GROT partition after the call to COPY_TABLE_STATS but then I did see statistics on a later run so I edited the post to reflect that.<\/p>\n<p>Why the difference? Well it&#8217;s not a COPY_TABLE_STATS issue and COPY_TABLE_STATS does <em>not<\/em> copy statistics for one subpartition and not the others. It just doesn&#8217;t make sense and never did but I somehow convinced myself it did. Here&#8217;s where those subpartition stats actually came from and you can walk through\u00a0 <a href=\"\/stats_5_6a.txt\">the script and output<\/a> I posted earlier to see this for yourself.<\/p>\n<p>(Note &#8211; I actually repeat steps 1 and 2 twice &#8211; once with _minimal_stats_aggregation set to it&#8217;s default value of TRUE and then with _minimal_stats_aggregation set to FALSE. But both runs have the same error, so you can refer to either)<\/p>\n<p>1) I create LOAD_TAB1 and re-use it 4 times to load 3 rows into each of the 4 new subpartitions using subpartition exchange. Here&#8217;s an example of one of those loads.<\/p>\n<pre>SQL&gt; TRUNCATE TABLE LOAD_TAB1;\n\nTable truncated.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100209, 'JUNE', 400, 'U');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100209, 'JUNE', 600, 'U');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100209, 'JUNE', 900, 'U');\n\n1 row created.\n\nSQL&gt; exec dbms_stats.gather_table_stats('TESTUSER', 'LOAD_TAB1');\n\nPL\/SQL procedure successfully completed.\n\nSQL&gt; ALTER TABLE test_tab1 EXCHANGE SUBPARTITION P_20100209_JUNE WITH TABLE load_tab1;\n\nTable altered.\n\nSQL&gt; ALTER TABLE test_tab1 MODIFY SUBPARTITION\tP_20100209_JUNE REBUILD UNUSABLE LOCAL INDEXES;\n\nTable altered<\/pre>\n<p>2) The subpartition stats now look like this<\/p>\n<pre>TABLE_NAME                     SUBPARTITION_NAME              GLO LAST_ANALYZED          NUM_ROWS\n------------------------------ ------------------------------ --- -------------------- ----------\nTEST_TAB1                      P_20100209_GROT                NO  22-APR-2010 11:24:12          3\nTEST_TAB1                      P_20100209_HALO                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_JUNE                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_OTHERS              NO  22-APR-2010 11:24:13          3\n<\/pre>\n<p>3) Now that I have the P_20100209 subpartition stats, I&#8217;ll add a new partition, populate the P_20100210_GROT subpartition with 10 rows using the same LOAD_TAB1 and subpartition exchange and then copy the stats from the previous partition.<\/p>\n<pre>SQL&gt; ALTER TABLE TEST_TAB1\n  2  ADD  PARTITION P_20100210 VALUES LESS THAN (20100211);\n\nTable altered.\n\nSQL&gt; TRUNCATE TABLE LOAD_TAB1;\n\nTable truncated.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 1000, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 30000, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 2000, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 10000, 'N');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 2400, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 500, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 1200, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 400, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 600, 'P');\n\n1 row created.\n\nSQL&gt; INSERT INTO LOAD_TAB1 VALUES (20100210, 'GROT', 700, 'Z');\n\n1 row created.\n\nSQL&gt; COMMIT;\n\nCommit complete.\n\nSQL&gt; ALTER TABLE test_tab1 EXCHANGE SUBPARTITION P_20100210_GROT WITH TABLE load_tab1;\n\nTable altered.\n\nSQL&gt; ALTER TABLE test_tab1 MODIFY SUBPARTITION\tP_20100210_GROT REBUILD UNUSABLE LOCAL INDEXES;\n\nTable altered.\n\nSQL&gt; exec dbms_stats.copy_table_stats(ownname =&gt; 'TESTUSER', tabname =&gt; 'TEST_TAB1', \n                                      srcpartname =&gt; 'P_20100209', dstpartname =&gt; 'P_20100210');\n\nPL\/SQL procedure successfully completed.\n<\/pre>\n<p>4) The subpartition stats now look like this <\/p>\n<pre>TEST_TAB1                      P_20100209_GROT                NO  22-APR-2010 11:24:12          3\nTEST_TAB1                      P_20100209_HALO                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_JUNE                NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100209_OTHERS              NO  22-APR-2010 11:24:13          3\nTEST_TAB1                      P_20100210_GROT                NO  22-APR-2010 11:24:10          3\nTEST_TAB1                      P_20100210_HALO                NO\nTEST_TAB1                      P_20100210_JUNE                NO\nTEST_TAB1                      P_20100210_OTHERS              NO\n<\/pre>\n<p>so it looks like COPY_TABLE_STATS copied stats for just the subpartition<br \/>\n I just loaded via subpartition exchange, but why didn&#8217;t it copy the<br \/>\nstats for the other subpartitions? The answer is that it <em>didn&#8217;t<\/em><br \/>\ncopy any stats for subpartitions. The stats for P_20100210_GROT were<br \/>\ncreated because we exchanged LOAD_TAB1 with that subpartition and <em><br \/>\nLOAD_TAB1 still had statistics from the previous time it was used<\/em>! <\/p>\n<p>If I had added a gather_stats call on LOAD_TAB1 immediately after populating it with data (as I did in earlier parts of the test), the stats would still have appeared on P_20100210_GROT, but NUM_ROWS of 10 would have highlighted that these were not copied stats, but stats from the load table. Alternatively, if I had dropped and recreated LOAD_TAB1 each time (or used a different load table entirely) and not gathered stats on it (as you wouldn&#8217;t if you were using COPY_TABLE_STATS) then I would have seen what I did at the outset &#8211; no subpartition stats at all! That would be a much closer version of how you might actually use this in production.<\/p>\n<p>A final alternative would have been to delete the inappropriate stats from LOAD_TAB1 when I reuse it. That&#8217;s the approach I&#8217;ve used in a very slightly modified version of stats_5_6a.txt that I&#8217;ve put <a href=\"\/stats_5_6a_fixed.txt\">here<\/a> (just search for delete_table_stats).<\/p>\n<p><strong><br \/>The Waffle<br \/><\/strong><br \/>As well as being an illustration of how easy it is to screw up tests when you start cutting and pasting sections from other scripts to (ha!) simplify things, this is a poster-child for discussing things with others. It was only when I sat down with my colleague Jari Kuhanen (excellent Oracle guy &#8211; looking for work soon &#128521;) to go over another aspect of these tests and he asked some pertinent questions, that the problem suddenly hit me. He didn&#8217;t need to tell me what the problem was &#8211; just him asking good questions made me think about it a different way and I suddenly knew what the issue was.<\/p>\n<p>I keep saying this, but one of my favourite aspects of blogging is making and fixing your mistakes in public. It&#8217;s often the mistakes that provide the real learning experiences and it&#8217;s also an extension of discussing your ideas with others. However, the down-side is that you can end up with an unholy, confusing mess of different, contradictory posts and it can be hard to judge what you should edit as a result. I never edit posts meaningfully without making it pretty clear I&#8217;ve done so and, although I&#8217;m tempted to tidy up this mess to make all the posts more consistent, I&#8217;ve decided I&#8217;d rather leave the meandering &#8216;story&#8217; as is. No promises on when this might happen but, when these posts are all done, I think there&#8217;s no avoiding a white paper about managing statistics on partitioned objects that will be coherent and correct. For now, let&#8217;s leave the mess as it is &#128521;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;m pretty self-concious of the amount of waffle that surrounds any technical content here, so let&#8217;s get the technical bit out of the way first, then the waffling can come later &#8230; I finally tracked down the mistake I didn&#8217;t make in part 6a, but thought I&#8217;d identified and fixed in part 6b! Here are&hellip; <a class=\"more-link\" href=\"http:\/\/orcldoug.com\/blog\/2010\/05\/07\/statistics-on-partitioned-tables-part-6d-copy_table_stats-a-light-bulb-moment\/\">Continue reading <span class=\"screen-reader-text\">Statistics on Partitioned Tables &#8211; Part 6d &#8211; COPY_TABLE_STATS &#8211; A Light-bulb Moment<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1602","post","type-post","status-publish","format-standard","hentry","category-uncategorized","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":1596,"url":"http:\/\/orcldoug.com\/blog\/2010\/04\/22\/statistics-on-partitioned-tables-part-6a-copy_table_stats-intro\/","url_meta":{"origin":1602,"position":0},"title":"Statistics on Partitioned Tables &#8211; Part 6a &#8211; COPY_TABLE_STATS &#8211; Intro","date":"April 22, 2010","format":false,"excerpt":"[Phew. At last. The first draft of this was dated more than two weeks ago .... One of the problems with blogging about copying stats was the balance between explaining it and pointing out some of the problems I've encountered. So I've broken up this post, with a little explanation\u2026","rel":"","context":"With 4 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1598,"url":"http:\/\/orcldoug.com\/blog\/2010\/04\/22\/statistics-on-partitioned-tables-part-6b-copy_table_stats-mistakes\/","url_meta":{"origin":1602,"position":1},"title":"Statistics on Partitioned Tables &#8211; Part 6b &#8211; COPY_TABLE_STATS &#8211; Mistakes","date":"April 22, 2010","format":false,"excerpt":"Sigh ... these posts have become a bit of a mess. There are so many different bits and pieces I want to illustrate and I've been trying to squeeze them in around normal work. Worse still, because I keep leaving them then coming back to them and re-running tests it's\u2026","rel":"","context":"With 9 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1568,"url":"http:\/\/orcldoug.com\/blog\/2010\/02\/28\/statistics-on-partitioned-tables-part-4\/","url_meta":{"origin":1602,"position":2},"title":"Statistics on Partitioned Tables &#8211; Part 4","date":"February 28, 2010","format":false,"excerpt":"In the last post I illustrated the problems you can run into when you rely on Oracle to aggregate statistics on partitions or subpartitions to generate estimated Global Statistics at higher levels of the table. Until there are statistics for all of the relevant structures then aggregation won't take place\u2026","rel":"","context":"With 3 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1563,"url":"http:\/\/orcldoug.com\/blog\/2010\/02\/22\/statistics-on-partitioned-tables-part-2\/","url_meta":{"origin":1602,"position":3},"title":"Statistics on Partitioned Tables &#8211; Part 2","date":"February 22, 2010","format":false,"excerpt":"In the last part, I asked you to trust me that true Global Stats are a good thing so in this post I hope to show you why they are, to make sure you don't kid yourself that you can avoid them. (Updated later - this is all on 10.2.0.4)Why\u2026","rel":"","context":"With 6 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1565,"url":"http:\/\/orcldoug.com\/blog\/2010\/02\/23\/statistics-on-partitioned-tables-part-3\/","url_meta":{"origin":1602,"position":4},"title":"Statistics on Partitioned Tables &#8211; Part 3","date":"February 23, 2010","format":false,"excerpt":"As soon as I'd committed my last post, I knew it wasn't what I'd hoped for and said as much to a couple of people before they'd read it. I knew it would probably just add to any confusion people already had about this subject (something I'm particularly keen to\u2026","rel":"","context":"With 11 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1562,"url":"http:\/\/orcldoug.com\/blog\/2010\/02\/17\/statistics-on-partitioned-tables-part-1\/","url_meta":{"origin":1602,"position":5},"title":"Statistics on Partitioned Tables &#8211; Part 1","date":"February 17, 2010","format":false,"excerpt":"If you've ever worked on large databases that use partitioned and subpartitioned tables, you'll be aware that there are significant challenges in maintaining up-to-date\/appropriate statistics. We've encountered a few problems at work recently and I decided it would be an idea to put together a series of posts covering the\u2026","rel":"","context":"With 11 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts\/1602","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/comments?post=1602"}],"version-history":[{"count":0,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts\/1602\/revisions"}],"wp:attachment":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/media?parent=1602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/categories?post=1602"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/tags?post=1602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}