{"id":1478,"date":"2009-03-30T12:00:00","date_gmt":"2009-03-30T12:00:00","guid":{"rendered":"http:\/\/orcldoug.com\/blog\/?p=1478"},"modified":"2009-03-30T12:00:00","modified_gmt":"2009-03-30T12:00:00","slug":"diagnosing-locking-problems-using-ash-part-2","status":"publish","type":"post","link":"http:\/\/orcldoug.com\/blog\/2009\/03\/30\/diagnosing-locking-problems-using-ash-part-2\/","title":{"rendered":"Diagnosing Locking Problems using ASH – Part 2"},"content":{"rendered":"

Some features in this post require a Diagnostics Pack license.<\/em><\/p>\n

I tried the graphical approach to tracking down the root cause of a locking problem in the last part<\/a>. Now let’s look at the underlying ASH data.
\nI ran the same test after bouncing my test instance to start from a clean slate. So, here it is again.<\/p>\n

Session 1 – logged in as TESTUSER<\/strong><\/p>\n

TESTUSER@TEST1020> select pk_id from test_tab1 where rownum=1 for update;\n\n     PK_ID\n----------\n      4051<\/pre>\n
Session 1 holds a lock on that row and I’ll use the PK_ID that was returned from the query to make sure I try to lock the same row in the next session.<\/p>\n
Session 2 – logged in as TESTUSER<\/strong><\/p>\n
TESTUSER@TEST1020> select pk_id from test_tab1 where pk_id=4051 for update;<\/pre>\n
Session 2 hangs, waiting for session 1 to release the lock. So I’ve just created ‘a locking problem’. Let’s get some details about the two sessions.<\/p>\n
SYS@TEST1020> select s.sid, s.state, s.event, s.sql_id, l.type, l.lmode\n\u00a0 2\u00a0 from v$session s, v$lock l\n\u00a0 3\u00a0 where s.sid = l.sid\u00a0 (+)\n\u00a0 4\u00a0 and s.username='TESTUSER'\n\u00a0 5\u00a0 order by s.sid;\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SID STATE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 EVENT\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SQL_ID\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TY LMODE\n---------- ----------- -------------------------------- ------------- -- -----\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 WAITING\u00a0\u00a0\u00a0\u00a0 enq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229 TM\u00a0\u00a0\u00a0\u00a0 3\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 WAITING\u00a0\u00a0\u00a0\u00a0 enq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229 TX\u00a0\u00a0\u00a0\u00a0 0\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 WAITING\u00a0\u00a0\u00a0\u00a0 SQL*Net message from client\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TX\u00a0\u00a0\u00a0\u00a0 6\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 WAITING\u00a0\u00a0\u00a0\u00a0 SQL*Net message from client\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TM\u00a0\u00a0\u00a0\u00a0 3<\/pre>\n
Session 1 is Session ID 157 and currently has the TX lock against the table (mode 6), whilst Session 2 is Session ID 150 and is waiting for the lock to be released.<\/p>\n
Session 1 – commits, to release the lock<\/strong><\/p>\n
TESTUSER@TEST1020> commit;\n\nCommit complete.<\/pre>\n
Session 2 – Is now able to acquire the lock, so locks the row and returns the result<\/strong><\/p>\n
\u00a0\u00a0\u00a0\u00a0 PK_ID\n----------\n\u00a0\u00a0\u00a0\u00a0\u00a0 4051<\/pre>\n
… and we can see that Session 2 (SID 150) now has the TX lock, session 157 is holding no locks and neither session is waiting for the other any more – they’re both idle (SQL*Net message from client).<\/p>\n
SYS@TEST1020> \/\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SID STATE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 EVENT\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SQL_ID\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TY LMODE\n---------- ----------- -------------------------------- ------------- -- -----\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 WAITING\u00a0\u00a0\u00a0\u00a0 SQL*Net message from client\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TM\u00a0\u00a0\u00a0\u00a0 3\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 WAITING\u00a0\u00a0\u00a0\u00a0 SQL*Net message from client\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 TX\u00a0\u00a0\u00a0\u00a0 6\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 WAITING\u00a0\u00a0\u00a0\u00a0 SQL*Net message from client<\/pre>\n
How does the ASH data look? I’ll query all ASH rows for TESTUSER sessions.<\/p>\n
SYS@TEST1020> select a.sample_id,a.sample_time,a.session_id,a.event,\n\u00a0 2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 a.session_state,a.event,a.sql_id,\n\u00a0 3\u00a0\u00a0\u00a0\u00a0 a.blocking_session,a.blocking_session_status\n\u00a0 4\u00a0 from v$active_session_history a, dba_users u\n\u00a0 5\u00a0 where u.user_id = a.user_id\n\u00a0 6* and u.username = 'TESTUSER';\n\n\u00a0SAMPLE_ID\n----------\nSAMPLE_TIME\n---------------------------------------------------------------------------\nSESSION_ID EVENT\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SESSION\n---------- -------------------------------- -------\nEVENT\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SQL_ID\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 BLOCKING_SESSION BLOCKING_SE\n-------------------------------- ------------- ---------------- -----------\n\u00a0\u00a0\u00a0 432483\n30-MAR-09 14.59.37.509\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ON CPU\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NOT IN WAIT\n\n\u00a0\u00a0\u00a0 432482\n30-MAR-09 14.59.36.509\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ON CPU\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NOT IN WAIT\n\n\u00a0\u00a0\u00a0 432481\n30-MAR-09 14.59.35.509\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 db file scattered read\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WAITING\ndb file scattered read\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NO HOLDER\n\n\u00a0\u00a0\u00a0 432480\n30-MAR-09 14.59.34.509\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 db file scattered read\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 WAITING\ndb file scattered read\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NO HOLDER\n\n\u00a0\u00a0\u00a0 432479\n30-MAR-09 14.59.33.499\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432478\n30-MAR-09 14.59.32.499\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432477\n30-MAR-09 14.59.31.499\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432476\n30-MAR-09 14.59.30.489\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432475\n30-MAR-09 14.59.29.489\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432474\n30-MAR-09 14.59.28.489\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432473\n30-MAR-09 14.59.27.489\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432472\n30-MAR-09 14.59.26.479\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\u00a0\u00a0\u00a0 432471\n30-MAR-09 14.59.25.479\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 150 enq: TX - row lock contention\u00a0\u00a0\u00a0 WAITING\nenq: TX - row lock contention\u00a0\u00a0\u00a0 0ut22yp7mh229\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 157 VALID\n\n\n13 rows selected.<\/pre>\n
Working my way from bottom to top (oldest data to most recent), I can see that Session ID 150 (that’s Session 2, remember) started off waiting for the row level lock and that lasted for 9 samples and then it started to read data (db file scattered read). I can also see that the session ID of the blocking session was 157. Excellent – just what I need to get to the bottom of this.<\/p>\n
But there’s a problem, isn’t there? Where are the samples for SID 157, which is the root cause of the problem? If we can’t see any samples for that session, how can we know what it was doing?<\/p>\n
That’s exactly what happened when I was demonstrating this last week and, once I’d pondered where the ASH data was for a little while, it hit me. The SQL statement that the blocking session executed was over so quickly, that the ASH sampling process missed it completely. The query started, selected the data and locked the row in less than a second, so it never appeared as an Active session as far as ASH is concerned. In fact, it was connected throughout the demo but was waiting on SQL*Net message from client and, as an Idle session, it wasn’t included in the ASH data. Yes, it was Idle, but it was also the root cause of this problem. Of course if I was looking at this problem while it was happening then I could use the BLOCKING_SESSION information from the blocked session’s ASH samples to look at V$SESSION, V$LOCK etc and work out what was going on, but that would be useless for diagnosing past locking problems with sessions that are now disconnected, which was the point of the demo.<\/p>\n
It was very obvious once it hit me and ASH is behaving exactly as designed and advertised. Just so long as you know that when ASH excludes sessions because they’re Idle at sample points, it might exclude the very sessions that are at the root cause of the problem.<\/p>\n
During the class, I suggested that this was a slightly artificial test because *all* that the blocking session had done was login, run a very quick SELECT FOR UPDATE and then sit Idle. In practise, most genuine user application sessions are likely to be generating enough activity that they would have appeared somewhere in the ASH samples. In fact, as I’ve been running tests while writing this, ASH captured the activity of the blocking session three or four times in a row, which was a little frustrating! 😉<\/p>\n
The design of ASH depends on the likelihood that there is enough activity to capture the essence of each session’s activity (and at very low cost) so maybe this isn’t such a big problem? To put it another way, the blocking session has hardly consumed any Database Time to speak of so it doesn’t really need to be optimised, does it? It’s the blocked session that needs to be ‘fixed’ and there are plenty of samples for that. So maybe we just shouldn’t use the data this way?<\/p>\n
If the blocking session had been active for a little longer – if it was experiencing it’s own bottleneck or was a long-running statement or transaction – then there would<\/em> have been ASH samples for it. The next post will look at an example of that.<\/p>\n","protected":false},"excerpt":{"rendered":"
Some features in this post require a Diagnostics Pack license. I tried the graphical approach to tracking down the root cause of a locking problem in the last part. Now let’s look at the underlying ASH data. I ran the same test after bouncing my test instance to start from a clean slate. So, here… Continue reading Diagnosing Locking Problems using ASH – Part 2<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1478","post","type-post","status-publish","format-standard","hentry","category-uncategorized","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":1480,"url":"http:\/\/orcldoug.com\/blog\/2009\/03\/31\/diagnosing-locking-problems-using-ash-part-3\/","url_meta":{"origin":1478,"position":0},"title":"Diagnosing Locking Problems using ASH – Part 3","date":"March 31, 2009","format":false,"excerpt":"Some features in this post require a Diagnostics Pack license.In the last part I described a locking scenario where the blocking session had only executed one SQL statement that was quick enough to avoid being sampled by ASH and is now inactive. As a result, there is no ASH data\u2026","rel":"","context":"Similar post","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1477,"url":"http:\/\/orcldoug.com\/blog\/2009\/03\/30\/diagnosing-locking-problems-using-ash-part-1\/","url_meta":{"origin":1478,"position":1},"title":"Diagnosing Locking Problems using ASH – Part 1","date":"March 30, 2009","format":false,"excerpt":"Some features in this post require a Diagnostics Pack license.There are few better illustrations of the utility of ASH data than the ability to detect and diagnose locking problems*, even after they've occurred. Because ASH is sampling all active sessions all of the time, you should have information both for\u2026","rel":"","context":"Similar post","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1481,"url":"http:\/\/orcldoug.com\/blog\/2009\/04\/09\/diagnosing-locking-problems-using-ash-part-4\/","url_meta":{"origin":1478,"position":2},"title":"Diagnosing Locking Problems using ASH \u2013 Part 4","date":"April 9, 2009","format":false,"excerpt":"Some features in this post require a Diagnostics Pack license.No sooner had I finished part 3 with some conclusions than I thought of another example I should have included and then someone else made a comment in an email which suggested another. (Thanks, JB!) \"It's probably worth pointing out that\u2026","rel":"","context":"Similar post","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1484,"url":"http:\/\/orcldoug.com\/blog\/2009\/04\/17\/diagnosing-locking-problems-using-ash-part-5\/","url_meta":{"origin":1478,"position":3},"title":"Diagnosing Locking Problems using ASH \u2013 Part 5","date":"April 17, 2009","format":false,"excerpt":"... and so it continues.Coincidentally, the subject of tracking down locking problems after they occurred cropped up on the Oak Table mailing list just after the last post. Several suggestions were offered but I think the nearest and most detailed suggestion was that proposed by Kyle Hailey, ex-Oracle and now\u2026","rel":"","context":"With 5 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1491,"url":"http:\/\/orcldoug.com\/blog\/2009\/04\/23\/diagnosing-locking-problems-using-ashlogminer-part-8\/","url_meta":{"origin":1478,"position":4},"title":"Diagnosing Locking Problems using ASH\/LogMiner \u2013 Part 8","date":"April 23, 2009","format":false,"excerpt":"So what about those SELECT FOR UPDATEs?I know that they\u2019ll generate redo entries and so something should appear in both log file dumps and the LogMiner output, but what exactly will appear? (This is all on Oracle 10.2.0.4)For this post I\u2019ll go back to the example from Part 4, where\u2026","rel":"","context":"With 4 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1488,"url":"http:\/\/orcldoug.com\/blog\/2009\/04\/22\/diagnosing-locking-problems-using-ashlogminer-part-7\/","url_meta":{"origin":1478,"position":5},"title":"Diagnosing Locking Problems using ASH\/LogMiner \u2013 Part 7","date":"April 22, 2009","format":false,"excerpt":"Picking up from the end of the last example, I immediately generated a log file dump as follows. (You'll need to look back at Part 5 to see that the ROWID, log file name etc. match up or just trust me that these steps came from a continuation of the\u2026","rel":"","context":"With 2 comments","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts\/1478","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/comments?post=1478"}],"version-history":[{"count":0,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/posts\/1478\/revisions"}],"wp:attachment":[{"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/media?parent=1478"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/categories?post=1478"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/orcldoug.com\/blog\/wp-json\/wp\/v2\/tags?post=1478"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}