SAP Startup Troubleshooting

 

1. IS DATABASE UP AND RUNNING?

This is an absolute prerequisite for SAP SYSTEM to start. In order to test this, operating system command R3trans -d can be executed (as SIDadm user). Any return code different than 0 means that DB cannot be reached by DIALOG work processes, and so SAP system will not initiate. The following return indicates that the database is up, running, and reachable by SAP work processes:

<hostname>:SIDadm > R3trans -d
This is R3trans version X.xx (release 742 - 18.11.14 - 20:14:09).
unicode enabled version
R3trans finished (0000).

-->There is NO POINT in moving the analysis further while the return code is different than zero.
-->In case the return code is different than zero, please involve your DBA/Network/OS teams.

2.IS SAP STARTUP AGENT (SAPSTARTSRV) RUNNING?

There must be ONLY ONE sapstartsrv process running for each instance. Examples for each platform below:

Examples

As an example, we will use a centralized system with the following 03 instances running on the same host (ASCS, PAS and DI). The following output should return for the operating system level command:

ps -ef | grep <SID> | grep sapstartsrv

(...) /usr/sap/<SID>/ASCS00/exe/sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_ASCS01_<hostname> -D
(...) /usr/sap/<SID>/DVEBMGS01/exe/sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_DVEBMGS00_<hostname> -D
(...) /usr/sap/<SID>/D02/exe/sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_D02_<hostname> -D

Profile pattern names returned with the command can be such as "START_<INSTANCE>_<hostname>" in older releases.
There can be another sapstartsrv process (as below) but it will play no part in the startup of the instance.

(...) /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile -D

When sapstartsrv service is not running you could run manually the above command for the desired instance
If the service doesn't start and nothing it's written in "sapstartsrv.log" it might happen the binaries are damaged in the local exe folder. In this case, has to be replaced

Starting & Stopping of SAPSTARTSRV

sapcontrol -nr 00 -function StartService/StopService 

3. IS THERE ANY SAP PROCESS ALREADY RUNNING?

After trying to start the instance (either through the SAP MMC, the "startsap" command or using "sapcontrol ... -function Start"), no error is returned. However, the SAP processes (Dispatcher - ABAP; jstart - Java) do not start. No trace file is updated either.

Verify whether any process is already in "GREEN" or "YELLOW" status (e.g., the IGS).

hostname:sidadm 40> sapcontrol -nr 33 -function GetProcessList
12.10.2015 20:15:45
GetProcessList
OK

name, description, dispstatus, textstatus, starttime, elapsedtime, pid
msg_server, MessageServer, GRAY, Stopped, , , 1234
disp+work, Dispatcher, GRAY, Stopped, , , 1111
igswd_mt, IGS Watchdog, GREEN, Running, 2015 07 31 11:59:15, 1:21:09, 2323
hostname:sidadm 41>

All processes must have the "GRAY" status. Otherwise, the SAP Startup Agent (sapstartsrv) will "ignore" the start command. Stop the instance first, so all processes are stopped. Then, start the instance again.


Logs checking Sequence :



     STDERR1 information about the start process of database system.
     STDERR2 information about the start process of message server.
     STDERR3 information about the start process of dispatcher. 

STDERR*  sapstart* files will be generated in work directory

Database starts

SAP checks if the database is available and if it is not it starts the database. No dedicated developer trace is created in the work directory as the database has its own logs.  If the database does not start correctly it should be visible within seconds.

Message server kicks in

The message server is the first part of the system to start, it handles the communications between the instances.  Only one message server is available per system, regardless of the amount of dialog instances available. The message server logs are kept on the dev_ms developer trace and if there is an issue with the message server it will be pretty obvious because you will not see a dev_disp trace.

Dispatcher next

Ahhh the Dispatcher!… well, the dispatcher dispatches (ha!), Nah, seriously… The dispatcher job is to receive the requests and direct them to an available suitable work process. During start-up you can see how the work processes are invited to start in the dev_disp developer trace. If something was to go wrong the only thing you will find in dev_disp is that the work processes died. Usually no reason is ever given for the failure to launch in this trace, but one dev_w* developer trace for each dialog process is created and populated in the process.

The dialog processes

This is where you’ll find the root of the problem 99.9% of the time. Why?…  Each dialog process has a dedicated memory area and a dedicated connection to the database layer. If something is going to go wrong it will most likely happen here. Each work process has its own developer trace dev_w* where you will find detailed information on the error.

So the logical troubleshooting sequence, in a nutshell is…

  • Check connection to the database with a quick R3trans –d command, it’s the quickest and simplest way to discard DB availability as the issue.
  • Go to the work directory and check the developer traces. If the dev_w* logs exist it means that the message server and dispatcher started and the issue is in the work process logs..
  • If the dispatcher and work process developer traces are not created then your issue is in the message server developer trace.

The next time you find a start-up issue I’m hoping that these simple steps might mean we can dispense with the crystal ball 😉


Reference https://wiki.scn.sap.com/wiki/display/SI/SAP+Startup+Troubleshooting+Guide+for+Netweaver+Application+Server




Comments

Post a Comment

Popular Posts

Interview Questions

Image
1. Startup profile from 7.3? Answer: Now the Start profile contents are merged with the Instance profile. With help of the new Instance, profile SAP processes are started and at the same time, instance-specific parameters are read. 2. SAP Application server connects with Database using database client. 3. Difference b/w profiles? Default.PFL - default profile Instance Profile - SID_DVEBMGS"nr"_Hostname location = usr/sap/SID/SYS/Profile 4. Below are the common errors in SAP transport management Return code 4:  Imported with warnings Generation of program, colomns or row missing . Return code 8 :  Imported with errors Syntax error Program generation error Dictionary activation error or method execution error Return code 12 :  Indicates Import Cancelled Program cancelled due to job Import cancelled due to object missing Import cancelled due to object not active Return code 18 :  Indicates Import Cancelled Due to system down while import Due to user expired during import Due to
Read more

EHP Upgrade (SUM)

Image
  Introduction: SAP Technical Upgrade is a periodic project that is implemented across companies to upgrade their SAP system to the latest released version. Most of the upgrade activities are done by the technical team, and functional consultants' role is limited and mostly confined to regression testing. Reasons for SAP Technical Upgrade : Since technology and businesses evolve, companies need to maintain a technological infrastructure that can support that change. This is the major reason why companies should undertake a technical upgrade of their SAP system. Of course, there are many secondary reasons, the significant one being the fact that like all software vendors SAP supports a released version of its software only for a limited period. Now we can go through the detailed phases included in the SUM upgrade process. latest SPAM version and Kernal also considered prerequisites  Prerequisites : 1. Check SYSTEM user is De-active, if yes, it should be activated. Passwords for th
Read more

SAP Logon Configuration(Load Balancing)

Image
Logon groups (or work groups) are configured to dynamically distribute the load being processed by the dialog work processes. Setting up logon groups helps in uniform distribution of the work load across the available instances. While logging on using a logon group, the ABAP message server is contacted to identify the instance with best performance statistics within the selected logon group . A report runs in SAP every 5 minutes which determines the load across each server and updates in the memory area of the message server. This information will be used by SAP GUI to determine the best instance to distribute the next user.   SAP Logon Process Let’s start with discussing an overview of SAP Logon process. High-level understanding of the process will help to explain details of SAP Logon configuration. SAP Logon Process Overview When user(s) accessing SAP system via  SAP GUI , the request is forwarded into ABAP dispatcher for the direct logon. If SAP GUI is using logon group, the logon w
Read more

HANA Installation & upgrade

Image
  Software Download: SAP HANA System Types : Ø   Single-host system - One  SAP HANA  instance on one host Ø   Multiple-host (distributed) system - Multiple  SAP HANA  instances distributed over multiple hosts, with one instance per host. SAP HANA Deployment Types : Tenant Databases : The tenant database deployment type makes it possible to run several SAP HANA instances on the same hardware in a production environment. Multiple Components on One Database (MCOD) : MCOD deployments are characterized by multiple applications on one SAP HANA system. SAP supports deploying and running multiple applications on a single SAP HANA production database  only  for packaged applications and scenarios listed on the "White List" included in SAP Note 1661202. Virtualized : SAP HANA systems can be run on virtual machines with restrictions to the hypervisor (including logical partitions). Multiple Components on One System (MCOS): MCOS deployments are characterized by
Read more

SAP HANA Backup

Image
  SAP HANA Persistence To ensure optimal performance, the SAP HANA database holds most of its data in-memory. However, it still uses persistent storage to provide a fallback in case of failure.   During normal database operation, data is automatically saved from memory to disk at regular savepoints . Additionally, all data changes are recorded in the redo log . The redo log is saved from memory to disk with each committed database transaction. After a power failure , you can restart the database as you would with any disk-based database. It then returns to its last consistent state by replaying the redo log since the last savepoints . Although savepoints and log writing protect your data against power failures, savepoints do not help if the persistent storage itself is damaged . To protect against data loss because of disk failures, backups are required . Backups save the payload (the actual data) of the data area and log area to different locations.   Types of Back-ups: 1.
Read more

SAP Transport Management System (STMS)

Image
TMS is used to move, manage and control copy development objects and customizing settings in an orderly fashion across SAP Landscape. STMS [SAP Transport Management System] Domain Controller–:  It is used to control the landscape. There will be only one domain controller in the landscape. Initially, it is set to development and later can be moved to on higher available system such as PRD/QAS/Pre7/ production but to keep things simple most of the cases we keep DEV as Domain Controller Member Systems–: The other systems are included in the domain referred to as member systems.  How to Configure   STMS Log on to the SAP system, which is decided to be the Domain Controller, in client  000  and enter the transaction code  STMS. If there is no Domain Controller already, a system will prompt you to create one. When the Transport Domain is created for the first time, the following activities happen in the background: Initiation of the Transport Domain / Landscape / Group Creating the user  TMS
Read more

SAP Kernel Upgrade in Distributed System

Image
What is the kernel of an SAP system? The kernel is a central program that acts as an interface between the SAP application and operating system. These files help startup the R/3 system, initialize the memory, create buffers and start managing the requests from users and effectively utilizing of hardware resources. The kernel is also responsible for starting and stopping all the application services like dispatcher, message server, collector, etc. Why Kernel Upgrade? SAP Kernel is the core of the application. Like all other applications, the Kernel contains the executable files (.EXE files for stating various processes in SAP). The Kernel is the heart of the operating system. It contains those files which are used to run every event in SAP. E.g.|: starting database, shutdowns of the database, starting sap, shutdown of sap, saposcol, to uncar the sap files etc. That's the reason why when a Kernel upgrade is done it means new versions of the various EXE files replace the older version
Read more

Set Up SAP HANA System Replication with hdbnsutil

Image
  Procedure:                                               SITE -A Enable system replication on the primary system(SITE A) as follows: In the Administration editor of SAP HANA studio, choose the  Configuration  tab and ensure that  log_mode  is set to "normal" in the  persistence  section of the  global.ini  file. Log mode  normal  means that log segments must be backed up. Log mode  overwrite  means log segments are freed by the savepoint (therefore only useful for test installations without backup and recovery). Do an initial data backup or create a storage snapshot. In multiple-container systems, the system database and all tenant databases must be backed up. As  <sid>adm  on the command line enable the primary for system replication and give it a logical name with the following command. The primary system must be online at this time: cd /usr/sap/<sid>/HDB<instancenr>/exe hdbnsutil -sr_enable --name=SITEA To check if the site has been successfully enable
Read more

SAP Client Copy

Image
Client copy means " transferring client-specific data"  within the same instance(SID) or between different instances(SID). Client copy can be performed with three different methods - Local client copy. Local client copy. Remote client copy. Client Import/Export. Prerequisites : Disconnect and lock dialog users. Suspend background jobs : This can be done using the report BTCTRNS1. The jobs will be released afterward using the report BTCTRNS2. Client Space check by using the report "RSSPACECHECK" in SE38. 1. Local Client Copy A local client copy copies between clients within the same SAP system. The local client copy must be initiated from the target client. The steps to perform a local client copy: Create an entry for the target client in the client maintenance table, choose transaction  SCC4 . In this example, client 101 will be created: Log on to the target client (client 101) as  SAP*  with the initial password pass (The user SAP* with the password pass is only ac
Read more