Skip Headers
Oracle® Fusion Middleware System Administrator's Guide for Content Server
11g Release 1 (11.1.1)

Part Number E10792-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to current chapter
Up
Go to next page
Next
View PDF

3.10 Finding Error and Status Information

Effective troubleshooting relies on the availability of useful, detailed information. The Content Server products provide various sources of information that may be helpful in the troubleshooting process.

This section covers the following topics:

3.10.1 Log Files

Content Server stores status information and errors in log files. Log files are used to register system events, with their date and time of occurrence. They can be valuable tools for troubleshooting, especially if verbose logging is turned on. Not only do logs indicate that specific events occurred, they also provide important clues about a chain of events that led to an error or problem.

Note:

When applied to process log output, verbose logging can quickly increase the size of a log file and possibly cause the content server to slow down. It is recommended that for process logs, verbose logging is only used when troubleshooting a specific issue. Regular content server logs do not have this issue with verbose logging.

Information is also captured in logs controlled by Oracle WebLogic Server using Oracle log APIs. The Oracle UCM interface provides access to these logs. For details, see "Viewing Log Information for Oracle UCM Content Server".

This section covers the following topics:

3.10.1.1 Log File Characteristics

The log files associated with Content Server have the following characteristics:

  • They are created only once each day at the time the first status, error, or irrecoverable error occurs.

  • No empty log files are generated.

Each log file contains the following columns:

  • Type: Specifies the kind of incident that prompted the log entry: Information, Error, or Fatal.

  • Time: Lists the date and time the log entry occurred.

  • Description: Describes the incident that occurred.

The log files are standard HTML pages and are maintained for each content server instance. Logs are kept in revolving file name format for a maximum of 30 files. When the 31st file is created, the oldest one is deleted. Therefore, log file names in Content Server bear no relation to the date they were generated. To find a certain day in the log file, view the index file in a browser and select that day's link. The file name is displayed in the browser's status bar (if it is enabled).

Tip:

Bookmark your log file pages. This will help you to troubleshoot problems, even if the content server is unavailable. Also, know where your configuration files are so you can find them if the content server is unavailable.

3.10.1.2 Accessing the Log Files

The log files of a content server are normally accessed from the Log Files folder in the Administration tray.

Note:

You must be logged into the content server as an administrator to be able to view the log files.

If, for whatever reason, you cannot view the log files from the Administration tray, you can also access them on the file system of the content server computer. The log files are located in the following locations:

Log Files Found in:
Content Server IntradocDir/weblayout/groups/secure/logs
Console Output Logs IntradocDir/bin/classname.log
Refinery IntradocDir/weblayout/groups/secure/logs/refinery
Archiver IntradocDir/weblayout/groups/secure/logs/archiver

3.10.1.3 Using Content Server Logs

The content server logs are listed by date and time. One file is generated for each day. Entries are added to the file throughout the day as events occur.

The following types of server log entries are generated:

  • Info: Displays basic status information. For example, status information is logged if the server is ready and waiting.

  • Error: Displays errors that occur but do not stop the software from functioning. For example, an error is logged if a user requests secure information that they are not allowed to access.

  • Fatal: Displays errors that stop the software from functioning. For example, a fatal error is logged if the content server cannot access the database.

To open a Content Server log:

To open a server log, complete the following steps:

  1. Ensure that you are logged into Content Server as an administrator.

  2. Click Content Server Logs on the Administration page or in the Administration tray's Log Files folder.

    The Content Server Logs Screen is displayed.

  3. Select the link that corresponds to the date and the time of the log that you want to view.

3.10.1.4 Using Archiver Logs

Archiver logs show information about imports, exports, and replications. The Archiver logs are listed by date and time. They are generated once a day when the first Archiver information status, fatal error, or error occurs.

The following types of archiver log entries are generated:

  • Info: Displays basic status information. For example, status information is logged when an export and an import starts and finishes.

  • Error: Displays user/administration errors that occur but do not stop the software from functioning. For example, an error is logged if there is no file information for a content item that you are trying to export.

  • Fatal: Displays errors that stop the software from functioning. For example, a fatal error is logged if the content server cannot access the database. Check the connection string, user name, and password.

To open an Archiver log:

To open an Archiver log, complete the following steps:

  1. Ensure that you are logged into Content Server as an administrator.

  2. Click the Archiver Logs link, found on the Administration page or in the Administration tray's Log Files folder.

    The Archiver Log Screen is displayed.

  3. Select the link that corresponds to the date and the time of the log.

    A table showing the type, date and time, and description of each action is displayed. It also includes the name of the content server instance that created the archive.

3.10.1.5 Inbound Refinery Logs

With the release of Inbound Refinery version 11gR1, all Refinery logging is accessed through the Inbound Refinery interface. For more information see the Oracle Fusion Middleware Administrator's Guide for Conversion.

3.10.2 Configuration Information

Content Server provides a Configuration Information Page that displays configuration information for a Content Server instance, which may be useful while troubleshooting a problem or working with the Oracle support organization. To access this page, click Administration in the portal navigation bar, then click Configuration for [Instance]. To display more information, click the link for each type of configuration information.

The following configuration information is presented:

  • Server information, such as name, description, and host filter.

  • Installation directories, such as the locations of the core Content Server software, the native file repository ('Vault'), and the Web-viewable file repository ('Web Layout').

  • Internet properties, such as the mail server and HTTP server names.

  • Database properties, such as the JDBC driver name and the JDBC connection string. Because Content Server uses an Oracle WebLogic Server data sore for the database connection, this information might not be displayed. If you do not see database connection information, check the Oracle WebLogic Server Administration Console.

  • Version and build information, such as the Content Server version and Java version.

  • License properties, such as the serial number and feature code.

  • Server options, which lists the current value of several server-specific options.

  • Content Security options, which specify what users can do with content.

  • Java properties, such as the JVM vendor and version.

  • Server components, which lists all components that are currently enabled for the content server instance.

    Note:

    Some options are specified during the software installation, while others are set using the System Properties utility.

3.10.3 System Audit Information

Content Server provides a System Audit Information Page for a Content Server instance, which may be useful while troubleshooting a problem or tweaking a server's performance. To access this page, click Administration in the portal navigation bar, then click System Audit Information.

The System Audit Information page has five sections:

3.10.3.1 System Audit General Information

The General Information section of the System Audit Information page provides the following:

  • Information regarding whether the system is receiving too many requests. If it is receiving too many requests, an e-mail is sent to the system administrator regarding load performance.

  • Information about the memory cache for the system, and is useful in troubleshooting any "out of memory" errors you may receive. This is important when running the content server with many users and a large quantity of data.

  • Information about which Java threads are currently running. This is useful in determining the cause of an error.

  • Information about database activity.

  • Listing of any audit messages.

To display more information, click the link for the type of configuration information.

Figure 3-15 System Audit General Information Screen

Description of Figure 3-15 follows
Description of "Figure 3-15 System Audit General Information Screen"

3.10.3.2 System Audit Localization Information

The Localization Information section of the System Audit Information page provides information on:

  • String key count

  • Whether the Localization system is using a string index

  • Localization test run time

  • Localization test lookups per second

Figure 3-16 System Audit Localization Information Screen

Description of Figure 3-16 follows
Description of "Figure 3-16 System Audit Localization Information Screen"

3.10.3.3 System Audit Tracing Sections Information

Tracing in Content Server can be activated on a section-by-section basis. Tracing for active sections is displayed on the Server Output Page. Section tracing is useful for determining which section of the server is causing trouble, or when you want to view the details of specific sections. Sections can be added by appending extra sections to create a comma separated list. A listing of the sections available for tracing, with brief descriptions, is available by clicking the 'i' next to the Tracing Sections Information heading. The wildcard character * is supported so that schema* will trace all sections that begin with the prefix schema.

Some tracing sections also support verbose output. Enable Full Verbose Tracing if you want to see in-depth tracing for any active section that supports it. See "Tracing" for more information.

Figure 3-17 System Audit Tracing Sections Information Screen

Description of Figure 3-17 follows
Description of "Figure 3-17 System Audit Tracing Sections Information Screen"

Important:

Any options set here will be lost when the content server is restarted unless you enable Save and click Update.

3.10.3.4 System Audit Cache Information

Content Server caches various items for quick access. The Cache Information section displays current information of three main caches:

  • Searches: This pertains to the number of searches currently being run, how many executed searches are currently in cache, and when the cache is emptied. These details are useful when troubleshooting any search related issues.

  • Schema: This lists details of any schema items currently in cache.

  • Buffer: This displays information about Java objects in cache and how much memory each object is using, which is reflected in the memory information under the System Audit General Information section. This information can be useful in pinpointing which object may be responsible for any memory leaks or other memory issues.

To display more information, click the link for the type of cache information.

Figure 3-18 System Audit Cache Information Screen

Description of Figure 3-18 follows
Description of "Figure 3-18 System Audit Cache Information Screen"

3.10.3.5 System Audit Configuration Entry Information

The Configuration Entry Information section of the System Audit Information page provides information on:

  • Number of environment keys

  • Number of overwritten config values

  • Number of ignored settings

  • Number of removed settings

To display more information, click the link for the type of configuration entry information.

Figure 3-19 System Audit Configuration Entry Screen

Description of Figure 3-19 follows
Description of "Figure 3-19 System Audit Configuration Entry Screen"

3.10.3.6 System Audit Component Report Information

The Component Report Information section of the System Audit Information page provides the following information for components on a content server instance:

  • Location: pathname for the component in the instance

  • Version: date, build, and revision

  • Status: current status of the component (Loaded or Skipped)

  • Reason: an explanation of the component status

To display more information, click the link for the component name.

Figure 3-20 System Audit Component Report Screen

Description of Figure 3-20 follows
Description of "Figure 3-20 System Audit Component Report Screen"

3.10.3.7 Server Output Page

The server output page displays the console output of Content Server. This is the same information that is located in the DomainHome/ucm/cs/bin/classname.log file. It includes information pertaining to all the sections selected for audit tracing in the System Audit Tracing Sections Information. To access the Server Output page, click View Server Output on the System Audit Information page.

Figure 3-21 Console Output Page

Description of Figure 3-21 follows
Description of "Figure 3-21 Console Output Page"

3.10.3.8 Localization Audit Page

The Localization Audit page is accessed by clicking Localization Auditing on the System Audit Information page. It displays information regarding the availability of localized variables for the Content Server user interface, and is useful in determining if any custom metadata field labels or other customized Content Server text requires localization. Clicking Show in the Stack Trace column displays the generated Java exceptions. Localization auditing is not persistent and must be started and stopped when using it.

Figure 3-22 Localization Audit Page

Description of Figure 3-22 follows
Description of "Figure 3-22 Localization Audit Page"

3.10.4 Tracing

You can activate Content Server tracing to display detailed system information that may be very useful for troubleshooting and optimizing system performance. There are two options:

3.10.4.1 Server-Wide Tracing

Server-wide tracing is used to view activities throughout the system.

There are two ways to activate server-wide tracing. To activate tracing from the Administration page, complete the following steps:

  1. Ensure that you are logged into Content Server as an administrator.

  2. Click the System Audit Information link in the Administration tray.

  3. Enable Full Verbose Tracing to see in-depth tracing for any active section that supports it.

  4. Specify the traces to activate.

  5. Click Update.

  6. Click View Server Output.

    Tip:

    Tracing options are lost on system restart. To ensure your settings are retained after restarting the content server, enable Save before clicking Update.

To activate tracing from an applet, follow these steps:

  1. Start an administrative applet.

  2. Select Options and then Tracing.

  3. Select Server tracing.

  4. Select the tracings to activate or all and click OK.

The following tracing options are available. Additional tracing sections can be displayed in the list if components are added.

  • applet: This trace contains result sets from initialized applets, such as the Configuration Manager or User Admin.

  • archiver: This trace provides information about archiving activities, including the reading and writing of archiver data files and the time the activities were initiated and finished.

  • archiverlocks: This trace provides information about the locks put on files during archiving activities, including time initiated.

  • chunkedrequest: This trace displays the messages and headers that are created when large requests are 'chunked' into smaller requests.

  • docprofile: This trace displays the computation of content profiles, specifically the evaluation of the rules that determine which fields are labels, hidden, and so on.

  • encoding: This trace provides information about encoding transformations that have occurred and the activities where encoding occurred.

  • filelock: This trace displays information about short-term system locks put on directories (during activities like archiving, for example) with a focus on collisions that occur and time outs.

  • filelonglock: This trace displays information about the creation, removal, and maintenance of long term locks imposed by the system.

  • filequeue: This trace displays information about accesses to a file queue.

  • indexer: This trace displays information about index functions that occur when the database is updated, including the steps taken to update the index and the time elapsed for each step.

  • indexermonitor: This trace provides a brief summary of automatic index activities, including time started and ended.

  • indexerprocess: This trace displays information about a manually launched index process and indicates if the process terminated properly.

  • localization: This trace displays information about localization usage and activities.

  • mail: This trace describes mail sent by the content server.

  • pagecreation: This trace displays information about the creation of displayed pages, including the server thread and the time taken to generate the page.

  • requestaudit: This trace provides summary reports on service requests, including the elapsed time for the requests and the number of requests made.

  • scheduledevents: This trace provides a list of hourly or daily background scheduled events.

  • schema: This trace provides information about schema publishing (tables and views published as .js files) and caching (tables cached into Content Server memory).

  • searchquery: This trace displays information about recent searches, including the fields used to search on and the order of sorting for results.

  • socketrequests: This trace displays the date, time, and thread number of socket requests and the actions during the request.

  • system: This trace displays internal system messages, such as system socket requests and responses.

  • systemdatabase: This trace provides information about database activities, including queries executed, index updates, threads used, and time initiated.

  • transfermonitor: This trace displays information about the archiver and the batch file transfer activities.

  • userstorage: This trace describes the access of external user repositories, including what actions were taken during access.

  • workflow: This trace displays a list of metadata on content items going through workflow, including document title and revision number.

    Note:

    To facilitate international support, most tracing messages are in English and do not have translations.

3.10.4.2 Applet-Specific Tracing

For applet-specific tracing, the output goes to the browser Java console. To perform tracing by applet, complete the following steps:

  1. Start the administration applet to be traced.

  2. Select Options and then Tracing.

  3. Make your selections, and click OK. The output is directed to the browser Java console.

Figure 3-23 Applet-specific Tracing

Description of Figure 3-23 follows
Description of "Figure 3-23 Applet-specific Tracing"

3.10.5 Environment Packager

The Environment Packager is a diagnostic tool. It creates a zip file of the desired state directories, log files, and other component and resource directories.

To create an environment zip file, complete the following steps:

  1. Ensure that you are logged into Content Server as an administrator.

  2. From the Administration tray, click the Environment Packager link. The Environment Packager Page is displayed.

  1. Select which parts of the environment should be packaged.

  2. When you are ready to create the environment zip file, click Start Packaging.

    A message is displayed while the zip file is being built, with a link to the zip file. The packaging process may take several minutes. The zip file link will not be available until the process has finished.

    Note:

    The packaged zip is named server_environment_*.zip. While Content Server builds the packaged zip file, it will be located in IntradocDir/vault/~temp. When the build of the zip file is complete, it is moved to IntradocDir/weblayout/groups/secure/logs/env.

3.10.6 Content Server Analyzer

Content Server Analyzer is a utility that enables you to confirm the integrity of the Content Server repository components, including the file system, database, and search index. It can also assist system administrators in repairing some problems that are detected in the repository components.

The Content Server Analyzer utility enables system administrators to do any of the following:

  • Confirm the accuracy of synchronization between three important Content Server database tables (Revisions, Documents, and DocMeta).

  • Confirm that the dRevClassID and dDocName fields are consistent across all revisions of content items.

  • Determine if the file system (native and Web-viewable file repositories) contains any duplicate or missing files.

  • Ensure the accuracy of synchronization between the search index and the file system.

  • Ensure the accuracy of synchronization between the search index and the Revisions database table.

  • Ensure that the file system contains all necessary files.

  • Remove duplicate files from the Content Server repository either permanently or provisionally by moving them into the logs/directory.

  • Produce a general report on the state of content items in the content server.

The method to start the Content Server Analyzer depends on the operating system:

  • Windows: Select Start, then Programs, then Content Server, then Instance_Name, then Utilities and then Content Server Analyzer.

  • UNIX: Change to the DomainHome/ucm/cs/bin directory and run the Content Server Analyzer program.

3.10.7 Using Content Server Analyzer

This section describes the following Content Server Analyzer tasks:

3.10.7.1 Accessing the Content Server Analyzer

To display the Content Server Analyzer, use one of the following methods:

  • Windows: Select Start, then Programs, then Content Server, then instance_name, then Utilities and then Content Server Analyzer.

  • UNIX: Change to the DomainHome/ucm/cs/bin directory, type IdcAnalyze in a shell window, and press the RETURN key.

The Content Server Analyzer application is displayed.

3.10.7.2 Specifying a Custom Analyzer Log Directory

The logs/ directory is the default logging directory for the Content Server Analyzer. Analysis output files are written to this directory and extra files detected during a file system analysis process can be transferred here as well. Optionally, the default logs/ directory name and path can be changed as desired.

To customize the Analyzer log directory name and path:

  1. On the Content Server Analyzer: Configuration Tab, place the cursor in the Analyzer log dir field.

  2. Enter the desired directory path.

    During the next analysis process, the Content Server Analyzer automatically creates the specified directory or directories in the DomainHome/ucm/cs/bin/ directory hierarchy.

3.10.7.3 Invoking the Analysis Process

To invoke the analysis process:

  1. On the Content Server Analyzer: Configuration Tab, select and activate the desired options (checking the corresponding check boxes).

  2. Click Start Analysis.

    Note:

    If this is the very first time the Content Server Analyzer has been run, the output files in the logs/ directory are automatically created. On subsequent analysis processes, a confirmation message is displayed asking to overwrite the existing log file.
  3. Click Yes to overwrite the existing log file.

    The Content Server Analyzer: Progress Tab is displayed automatically.

    Note:

    If you click No, the analysis process is terminated and you are prompted to manually remove files from the logs/ directory before running the Content Server Analyzer again.

    A completion message is displayed when all of the selected analysis processes are finalized.

  4. Click OK.

    The results are displayed in the console area on the Progress tab.

3.10.7.4 Analyzing the Content Server Database

These options are used to check the integrity of the database columns. The available options enable users to examine the three tables that are used to store content item revision information (DocMeta, Documents, and Revisions). The DocMeta file is examined for extra entries that are not found in the Revisions table. Similarly, the Documents table is examined to verify that there are sufficient entries to correspond to the entries in the Revisions table.

Description of analyze_db_options.gif follows
Description of the illustration analyze_db_options.gif

Note:

The Check RevClassIDs and Clean database options are activated and selectable only when the Check database option is selected.

To analyze the Content Server database:

  1. On the Content Server Analyzer: Configuration Tab, select the applicable options.

  2. Click Start Analysis.

    The results are displayed in the console area on the Content Server Analyzer: Progress Tab. See "Invoking the Analysis Process" for information about the analysis procedure.

3.10.7.5 Analyzing the Content Server Search Index

These options are used to check the entries in the Revisions table to ensure that all of the documents that belong in the index are properly listed. Additionally, a check can be performed to ensure that there are no duplicate entries in the search index.

Description of analyze_index_opts.gif follows
Description of the illustration analyze_index_opts.gif

Note:

The csIDCAnalyzeCleanIndex option is activated and selectable only when the Check search index option is selected.

To analyze the Content Server search index:

  1. On the Content Server Analyzer: Configuration Tab, select the applicable options.

  2. Click the Start Analysis button (refer to "Invoking the Analysis Process" for information about the analysis procedure).

    The results are displayed in the console area on the Content Server Analyzer: Progress Tab.

3.10.7.6 Analyzing the Content Server File System

These options check the integrity of the file system (weblayout and vault file repositories). Using the information in the database, these options ensure that every file in the Revisions table contains accurate entries corresponding to the items in the file system. A check can also be completed to locate any extra files in the vault and weblayout file repositories.

Description of analyze_filesys_opts.gif follows
Description of the illustration analyze_filesys_opts.gif

Note:

The Delete, Safe delete, and Check for extra files options are activated and selectable only when the Check file system option is selected.

To analyze the Content Server file system (vault and weblayout file repositories):

  1. On the Content Server Analyzer: Configuration Tab, select the applicable options.

  2. Click Start Analysis.

    The results are displayed in the console area on the Content Server Analyzer: Progress Tab. See "Invoking the Analysis Process" for information about the analysis procedure.

3.10.7.7 Viewing the Analysis Progress and Results

The Content Server Analyzer: Progress Tab is displayed automatically when the Start Analysis button is clicked. The progress bars show when the Content Server Analyzer has completed processing the selected analysis options. The following image shows a partially finished analysis:

When the analysis process is complete, the results are displayed in the console area of the Progress tab. The results depend on what analysis options were selected. The following image of the console area shows the results from selecting database, search index, and file system options:

Note:

The Generate report option was not selected for this example (refer to "Generating a Status Report" for an example of the generated status report).

Figure 3-24 Example Console Display of Results

Description of Figure 3-24 follows
Description of "Figure 3-24 Example Console Display of Results"

3.10.7.8 Generating a Status Report

The status report generated by the Content Server Analyzer provides statistics about the content items in the repository. The status report output is displayed in the console area of the Progress tab.

To generate a status report:

  1. On the Content Server Analyzer: Configuration Tab, select the Generate report check box.

  2. Click Start Analysis.

    When the analysis process is complete, the status report information is displayed immediately following the standard analysis results in the console area of the Content Server Analyzer: Progress Tab.

3.10.7.9 Canceling the Status Report

The report generation feature can be suppressed after the analysis process has already started. To cancel the content item status report during the analysis process:

  1. During the analysis process, click Cancel on the Content Server Analyzer Application.

    You are prompted about canceling after the current task is finished.

  2. Click Yes to suppress the status report.

    The status report is not included with the analysis results that are displayed in the console area of the Progress tab.

3.10.8 Configuration Debug Entry

Content Server also provides a debugging configuration variable that, when set, contributes applicable diagnostic information. The configuration variable is named IsDevelopmentEnvironment, and it is set in the Content Server's configuration file (IntradocDir/config/config.cfg) during installation and when Content Server is updated. This entry does the following:

  • Defines whether Content Server should run in debug mode.

  • Enables a trace of script errors. If used as a parameter to a service call, script error information can be added to the bottom of the displayed page.

Another debug configuration variable is named AlwaysReportErrorPageStackTrace. When this variable is set, whenever an error occurs the stack trace is reported on the browser showing the Content Server user interface.

Note:

For further details refer to the Oracle Fusion Middleware Idoc Script Reference Guide.

3.10.9 Stack Traces

The stack trace enables you to see what threads are currently running in the Oracle Content Server. It is a useful troubleshooting tool that provides information about the threads and enables you to monitor the Content Server's processing.

For instructions to initiate a current stack trace for a Content Server instance, see Oracle WebLogic Server documentation.