Tracing in SDL Tridion 2013 SP1 - A walk in the park, or.. beach

Some time ago, a little birdie told me in SDL Tridion 2013 SP1, all windows services can be started in debug mode.

While I have not made the time to check this out, I have through experience and some focused searches discovered that tracing has been taken to yet another higher level.

Some of the logs below will be familiar from previous versions, however, there are a few surprises you won't mind finding out about.

Like a walk on the beach, where tracing back the same path you took to arrive where you are can be both nostalgic and new, here is a collection of traces to walk you from the Content Manager to the Content Delivery and back in case you need to revisit some points in time..

What	Where (log or trace file)	How (Config file & Steps)
CME Javascript/CSS	Browser Developer Tools (eg. Sources and Console tabs) IE Firefox Firebug Chrome	http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-9652A9DD-3E44-44ED-98C8-28A7FF955CA0 %TRIDION_HOME% WebRoot\Configuration\system.config 1. Disable JS/CSS minifiers <filters>     <filter for="Script" type="Tridion.Web.UI.Resources.Filters.JScriptMinifier, Tridion.Web.UI.Resources.Filters" enabled="never"> ... <filter for="Style" type="Tridion.Web.UI.Core.Configuration.Resources.Filtering.CssCompressorFilter, Tridion.Web.UI.Core" enabled="never" /> 2. Refresh browser 3. Can call GUI urls with mode=js 4. Check Sources, Console tabs in browser developer tools to set breakpoints and/or follow execution or variable values
IIS	%SystemDrive%\inetpub\logs\LogFiles\ <ID of Web site>	Usually enabled, if disabled, check how to enable depending on IIS version
CMS Events	%SystemRoot%\System32\Winevt\Logs\ Tridion.evtx %SystemRoot%\System32\Winevt\Logs\ Tridion Content Manager.evtx	Enabled by default, look for them in the Windows Event Viewer console
CMS Core Service	Own trace	%TRIDION_HOME%\bin\TcmServiceHost.exe.config http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-FE919074-22B4-4A00-BDB7-92EF0DC20C3C
CME Server Log / CMS WCF	%TRIDION_HOME%\web\WebUI\WebRoot\ Tridion.Web.trace	%TRIDION_HOME%\web\WebUI\WebRoot\bin\web.config http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-16AB0377-ECB7-4FCC-9EB3-128F75AF1E90
CMS Workflow Service	Own trace	%TRIDION_HOME%\bin\TcmWorkflowAgent.exe.config
CMS Search Indexer	Own trace	%TRIDION_HOME%\bin\TcmSearchIndexer.exe.config
CMS Batch Processor	Own trace	%TRIDION_HOME%\bin\TcmBatchProcessor.exe.config
CMS Publisher Service	TRIDION_HOME%\bin\TcmPublisher.exe.config Tridion.ContentManager.Publisher.log	To run publisher in debug mode from command line: not supported, not documented, unexpected results: dies when user running it logs off? -only to be done in Production envs under Customer Support request publisherpath>TcmPublisher.exe /debug ---------------- To have publisher log to file: - In MMC set "Publisher settings" - "Activate logging to publisher actions" option to "true", don't need to restart COM+ or Publisher - Check "c:\program files\tridion\log\Tridion.ContentManager.Publisher.log" - (It doesn't track when rendering in TemplateBuilder template tool) - Publisher log configurable in : \Tridion\bin\TcmPublisher.exe.config
CM Transport	c:\tridion\log\cd_transport_<date>.log	%TRIDION_HOME%\config\logback.xml <property name="log.level" value="ALL"/> or <property name="log.level" value="TRACE"/>
CD Transport	c:\tridion\log\cd_transport_<date>.log c:\tridion\log\cd_core_<date>.log	%TRIDION_DEPLOYER_HOME%\bin\config\logback.xml http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-DF88578C-8B32-4933-A6F5-939DA3275D95
CD Deployer	c:\tridion\log\cd_deployer_<date>.log c:\tridion\log\cd_core_<date>.log	Normal Trace and Hibernate Trace %TRIDION_DEPLOYER_HOME%\bin\config\logback.xml <property name="log.level" value="ALL"/> or <property name="log.level" value="TRACE"/> ----------------------------------------------------- <root level="ON">      <appender-ref ref="rollingCoreLog"/> </root>
OutBoundEmail	C:\Program Files (x86)\Tridion\Log\OutBoundEmail.log	%TRIDION_HOME%\config\OutboundEmail.xml <Log>         <Folder>C:\Program Files (x86)\Tridion\log</Folder>         <Level>5</Level> </Log>
UGC	%TRIDION_HOME%\web\WebUI\WebRoot\ Tridion.Web.Trace	1. Backup %TRIDION_HOME%\web\WebUI\WebRoot\Tridion.UGC.Model.dll 2. Replace with %TRIDION_HOME%\web\WebUI\WebRoot\trace\Tridion.UGC.Model.dll 3. Reset IIS

As with any logging in general, beware lots and lots of data will be accumulated in a very short amount of time.

So

- focus your actions

- limit the number of users in the system during tracing

- check there is enough memory available

- check there is enough storage available

and.. don't forget to turn off the traces when done investigating!

Happy tracing!

Quantifying Rambles

Recently I've come across a concept which oddly I had partaken as a matter of fact, part of life, encountered in my daily routines and more so pretty much everywhere else I turned around.

I'm talking about the myriad of little devices tracking here, there, everywhere, from time to calories, from steps we take to where we are and where we go, and the concept behind it I've come to know is called QS or quantifying self.

Being both a sci-fi fan and a techy, my mind inventoried a quick little list of hot wearable technology:

- watches

- cell phones

- fit bands

- tablets

- chips

.. and then raced ahead on what else is in prototype mode and/or about to hit the market..

- Google (eye)glass(es)

- electronic clothing

- tracking rings and fit bracelets

- more sophisticated watches

- portable SixthSense projections

- self-aware devices fixing themselves or communicating information to each other via 'speckled' computing

.. all capturing, analyzing and measuring themselves or any one person in a given context or scenario.

So then my train of thought took a light curve and I began wondering what life would be like soon enough, what I'd be spending my time using in my working environment.  How would I handle all this semantic data physically, and what would connect me to the software I work with?  The concept is not new, but what's changing is the hardware.

From quantifying self to quantifying content, I think there will emerge a more proprietary standard platform of software to enable communication between devices and even between devices and people!  We are probably already seeing some of it reflected in the meta of responsive design, where HTML is now changing to account for context and descriptive attributes in its format as a language, founded on the principles of XML.

Already today, I can connect to SDL Tridion using my phone, my tablet, my laptop and so why not in the near future, from anything else I might happen to wear.  For instance, I could be walking in a botanical garden wearing the new Google glasses and presented with more in-depth information about the flowers I'm looking over feel inspired to create a new marketing campaign spun from African orchids.  I could press my bracelet and open a projection of the GUI which by now may have morphed into a semantic taxonomy of sorts, ready for me to begin quantifying content, tip-tap, quick selections, move on to typing a few key sentences and off it goes, workflow-ed for approval and review, prepped for publishing in a cloud.

Perhaps another even shorter path to my web_of_things Content Manager could come directly prepped with all the data, and I would simply instruct it at a touch to insert itself in the software for other tracking, management, analysis and use.

The technology is in motion, like spring, it has only just begun to blossom.  Soon enough intelligent-data-aware objects will drive much of what we are and do; set the wheels in motion and start thinking about what this might mean to you.  Start classifying your content and quantifying your future business.

Troubleshoot the SDL Tridion Deployer Service - Part 2

Here we are as promised, in part 2 of troubleshooting the Deployer I continue to follow more ways to figure out what could be causing certain deployment woes.

Let's take a look at one of the most important lines in the cd_deployer_conf.xml file

<Queue>
        <Location Path="c:\tridion\incoming" WindowSize="20" Workers="10" Cleanup="true" Interval="2s"/>

1. Cleanup set to false - an excellent source of data is letting the deployment package remain in the folder where it was created, instead of having it removed as part of cleanup.  This is possible by setting the attribute Cleanup="false" in the cd_deployer_conf.xml file.  This package then can be inspected to match what is being moved by the Transport service and whether it matches expectations.  Remember to set this back to "true" when done debugging as files may collect here quickly and take up unnecessary space.

Also, this package can be dropped in the incoming folder of a working deployer to ensure there is nothing wrong with the package itself.

2. Is the incoming folder clean - another great checkpoint, having packages left over may cause the deployer to continue and try processing what it finds in this folder, possibly being hung by a transaction it can no longer distribute correctly

3. Increase/decrease WindowSize - use this to control how many packages the deployer allows in the queue and should handle at one point , thus helpful for cases when items are showing as "Waiting for Deployment".

4. Increase/decrease Workers - change this setting to define the number of deployer threads, also helpful when seeing "Waiting for Deployment".  Use it together with WindowSize to fine tune performance - how many deployment threads at a time should handle a given number of packages, carefully, though, tread as logically as Mr. Spock because giving the deployer threads too many packages too handle, will cause whatever cannot be processed at one time to go on hold - in "Waiting for Deployment", and typically this is seen when the WindowSize is more than double the number of Workers

5. Type of deployer and associated problems - there are several types of deployers, beware which one is in use, and possibly issues surrounding it based on type

• Windows Service - think start/stop/restart/dependencies only works for non HTTP/HTTPS transport protocols
• HTTP/HTTPS - think IIS, web.config, upload module, authentication
• • Test URL: http://HOST:PORT/HTTPUpload.aspx
• Java - think Tomcat, web.xml, web-inf classes, war file
• • Test URL: http://HOST:PORT/cd_upload/httpupload

6. Is the issue due to another service, are the Transport and Publisher in trouble, and the Deployer is stuck because of that instead?

7. Are there any extensions or custom modules installed on the deployer ? (http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-894AB0F0-313C-45A3-B8FC-8A02DA894C93)

Has the default behavior of the deployer been modified?

Take note of entries not part of the sample deployer configuration file available from install, such as the example below:

<Module Type="CacheFlusher" Class="com.tridion.examples.CacheFlusher">

8. Multi-threading - is the number of deployer threads too high compared to the number of the server's CPU cores?

9. WindowSize = 0  - if the window size is set to 0 (default), the Deployer sets no limit on the number of items in transit, guess what will happen based on what we talked in the previous points!

10. Constraint violations - may occur for different reasons, varying from table keys, to amount or type of content inserted in the Broker, and probably best addressed by contacting Support.  Prevention is always good medicine so periodically check the hotfixes section on SDL Tridion World for issues already discovered and addressed.

Until next time, happy troubleshooting!

Troubleshoot the SDL Tridion Deployer Service - Part 1

The Deployer Service is step 3, the finalizing step in the SDL Tridion publishing process.

In working with the Deployer, I've come across different reasons something could go sideways, and so in time, several best practices, a certain analysis pattern and some specific areas of interest began to emerge in the way I look at an issue here. 

The purpose of the Deployer service is to complete the publishing process by taking the transport package available in a specified location, usually known as the "incoming" folder, and processing its instructions.  This is done in accordance with the deployer's configuration (a combination of deployment information and data storage options).

It's hard to pinpoint where I begin, hence the brainstorming below.  Typically if there is a specific error I consider that the starting line, and so the next logical step takes me over to the logs, but more often than not, I feel better equipped to work on figuring out an issue if I know the architecture of the system involved.  It also helps to pay close attention to the "behavior" exhibited.  

Time and time again the first question that surfaces especially in situations where seemingly unexplainable ghosting behavior is observed becomes about the number, type and setup of deployers.  These have actually become my favorites: explain the unexplainable!

As I begin the analysis, I make sure I have the following 3 things: 

- the item id

- the publish queue transaction id I will need to trace along

- the timeframe in which the issue occured

.. and of course, as much clarity as I can muster about the symptoms experienced.

1. Are there 2+ (multiple) deployers referencing the same incoming folder - the behavior encountered is somewhat mystical in nature - records will not arrive at their destinations, transactions will not show up in the logs, and so content will not be available on the respective websites.

This can happen should 2+ deployers be configured to monitor the same incoming folder, a racing condition can occur, where the two deployers begin to compete and pickup transports depending on availability.

2. Are licenses correct - validate the path specified for configuration files, and if necessary, remember to request licenses ahead of time from SDL Customer Support.

3. Are there multiple destinations in the target - this is important to be aware of proper permissions and correct paths or urls. 

4. Is there a "rogue" deployer running somewhere - another mystical behavior, the clue here is to watch if transactions just "disappear" and don't show up in known logs at all, with some other deployer running and picking up jobs.  An interesting place to check in this case is the QUEUE_CONSUMERS table, look for machines activated which should not be. Deactivate what should not be in use anymore.  Note the deployer itself is not registered in the QUEUE_CONSUMERS table, the reason this is mentioned is to serve as a hint for what machines may possibly be carrying it which should not.  It is possible to have a deployer installed and running from a Content Manager server, although the preferred setup for scalability and performance is to decouple Content Delivery from Content Management.

5. Check what hotfixes are installed - perhaps the error message is a clear indication of an issue already fixed, an excellent source for this is available on http://www.sdltridionworld.com, I login and visit the Support section to check what hotfixes are posted and/or have been installed.

6. Check cd_core and cd_deployer logs - I look here to trace a specific transaction or item, best bet is to put these in DEBUG or TRACE mode, however beware logs will grow quite rapidly depending on the volume of data published.  Also search for "ERROR"s or "WARN"ings. 

7. Are there jars missing - probably one of the lengthier checks, but this can be scripted for comparison against a working site or the installation folders.  Typically the errors for this will be "Class not found", or "java.lang.NullPointerException", and so I follow the stacktrace to determine what call may have originated the issue and therefore what is missing.

8. Is the storage configured correctly -  beware what is defaulted and what becomes inherited.  Beware what is going to the file system or to the database.  Beware what is cached.  Beware the database connections information is up to date and test connections are working.

9. Is the cd_deployer_conf.xml correct - is the xml formatted correctly, are locations and paths correct?

10. Setup hibernate logging - If errors seemingly database related  are seen, this logging can trace most calls to the database.  This type of logging comes with a high warning: information will be logged quickly so use it to replicate the issue fast and when ready, turn off promptly.  Check Tridion StackExchange for steps on how to enable it and detect what hibernate entries look like.

By now, I may have solved the issue, or otherwise I am in possession of quite some good information and ready to perform more indepth analysis.  If I still cannot determine root cause or fix the issue, I continue my research further to increase the information collected.

In Part 2 of the article, I follow on with more clues, tips and fixes.

How to Setup Classic Editing Workflow in SDL Tridion 2013 SP1

SDL Tridion 2013 SP1 introduced two new approaches related to workflow functionality.  As such the product now provides four types of workflow for implementations:

• Bundle Workflow
• Task Workflow
• Programmatic Workflow
• Classic Editing Workflow 

The last type of workflow is also known as legacy workflow these days, having been superseded by the Bundle and Task workflows.  I recently did some work using this type of workflow and while getting started, I made my way across the following steps I thought would be useful to keep track of.  As interesting and beneficial the Bundle and Task workflows are, perhaps some implementations will decide to retain legacy workflow for a while longer, so here we go.

1. Classic Editing / Legacy workflow does not install by default, so the previous default Component and Page Process will not be available in the box.. out of the box

2. Workflow diagrams are created and edited with Visio.  Installing Microsoft Office Visio 2013 is preferred, otherwise Visio 2010 is also supported (although you may need to install updates the first time the client runs)

3. Install Visio Workflow Designer Client from Content Manager\Additional Products\Visio Workflow Designer is necessary as well.  This is the extension applied to Visio which enables connectivity between Visio and the Tridion Content Manager.  Once this runs, the next time you open Visio, you will notice and recognize several new Tridion icons appear in Visio, under the SDL Tridion Workflow Shapes section.

4. To bring in previos classic workflow processes, you can export Tridion Visio diagrams from 2011 SP1 locally and import them in 2013 SP1 using Visio Tridion functionality

5. When working on step 4 above, if you get this screenshot on the 2013 system, you have not installed the actual legacy pack 

"Workflow Script Type with Name 'VBScript' was not found in the configuration file"

Run the installer and make sure the following option is set:

SDLTridion2013SP1CM.exe LEGACY_VISIBLE=TRUE

6. Once all this is in place, proceed to associate the newly imported workflows as needed in Schemas for Components, Structure Groups for Pages or Publications for Templates.  

Should some of the concepts need refreshing, review them again, and remember, workflows were meant to introduce simplicity as well as security by making sure that specific users go through steps in a specific order.  Always strive to simplify a given workflow and the results will be rewarding. 

SDL Tridion 2013 SP1 - How to Setup a 2nd Broker

Whenever I've come across installing Tridion, I've always installed at least two broker databases, and lately three if you count the Session Preview broker as well.  The first broker is for Staging websites, while the second is used for Live sites.

As is the case with a repetitive action, the more you do it, the faster and better you become at it.

Here are several pseudo-steps as an easy recipe to get you through this setup fast as fast can be in Tridion 2013 SP1.

1. Install the actual database and know the database credentials ahead of time*: run the Powershell script

& '.\Install Content Data Store.ps1' 

2. Name your broker something different than what is in the database already (eg. Tridion_Broker_Staging)

3. Update your (second) Deployer cd_storage_conf.xml with the new broker information and configure how you want different item types to be handled.  A second separate deployer is required to write to the new database.

4. Update (another) corresponding website cd_storage_conf.xml to show content from the second broker.  A separate website is necessary as well to show content stored in the new broker.

5. Loop back into the Tridion Content Manager Explorer and create the necessary [Staging] target type and publication target(s).

Specify the deployer information based on type installed and apply the publications you want to store content in the new broker.

.. and voila, the second broker should be active and receiving content when you publish.

Happy brokering!

---

References (login required):

*http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL%20Tridion%20full%20documentation-v1/GUID-396764E5-8C75-4444-B12A-3EA8ECF4A3C1

Quick Check List for SDL Tridion 2011 SP1 Personalization and Profiling Setups

In SDL Tridion 2011 SP1, setting up pesonalization and profiling can be somewhat challenging given the range of configuration touch points spans both the Content Manager and the Content Delivery systems.

If you are looking for an installation, configuration and implementation path, you could likely follow these steps (depending on whether this is .NET or Java based): 

1. Check the Target Publishing Language in the Publication Target is dynamic (JSP/ASP/REL)

2. Check the cd_deployer_conf.xml file includes TCDLTransformer class, for PageDeploy for example

3. Update the cd_storage_conf.xml (or cd_broker_conf.xml if still using that) to enable Item Types for the elements you want to track (eg. PageTracked, Timeframe etc)

4. Update the cd_wai_conf.xml to represent your website and settings for your personalization and profiling requirements such as host, cookies, tracking info

5. Update the web.config to support the httpModule TridionWAIHttpModule

6. Restart Content Delivery services and web servers for changes to take place

6. Proceed to enable item tracking such as Component and Page Template tracking in the CMS using template building blocks

7. Check if tracking is working after publishing your changes from the CMS by viewing content from a browser

8. Continue to implement personalization and profiling requirements using CMS functionality and API calls and test they function as expected after publishing and viewing pages from a browser 

Additional references for Personalization and Profiling can be found below.

Setting up/Configuring Reference:
http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL_Tridion_2011_SPONE/reference_CD033408F5CD4A119BDA616851A10B3F

Setting up P&P (WAI) for a .NET application
http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL_Tridion_2011_SPONE/task_FF19F86A63E246F793D64495849EDEDF

Implementing:
http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL_Tridion_2011_SPONE/idheading-158522992

Reference API:
http://sdllivecontent.sdl.com/LiveContent/content/en-US/SDL_Tridion_2011_SPONE/pp_concept_B72F4D64713F4682B61C838A013DF8F8