“ORA-01081: cannot start already-running ORACLE” While creating a new Database

articles: 

Hi Geeks,

When I was doing a research on Oracle Database creation, through DBCA, last week, I came across a very interesting scenario which I couldn’t stop sharing with you :)

I have a cluster of machines (don’t confuse it with a RAC cluster but just a group of machines required for my application deployment) with 11.2.0.2 installed and a Database instance running.

I got a requirement to remove everything accumulated in the machine as a result of the Oracle installation and the Database creation: The installation directories, the datafiles, oratab entries etc…

After removing all these things, I need to do a fresh installation again, create a Database and make the system running as it was before the reset activity. But the main point to consider here is, we have to do this on a cluster of machines and as quickly as possible.

So, I thought of removing the Databases with DBCA and do a ‘brute-force’ removal of all the physical files. I’ve noticed that the DBCA is taking at least 3-5 minutes for each Database to remove so in a cluster of 10 Databases it’ll add a burden of 30-50 minutes unnecessarily.

So, I’ve decided to remove everything related to Oracle from each machine in a brute-force way and start the installation and Database creation steps.

After finishing up the installation and receiving a zero return value from DBCA I thought of cross checking the things and observed that the Database is not created properly; no datafiles were created even but just an entry in /etc/oratab and some other stuff despite the success reported by DBCA…

To my bad, of course good, after seeing the DBCA logs, I found that the Database creation was failing with a very weird error.

ORA-01081: cannot start already-running ORACLE

I’m almost surprised on seeing this message!!!

As we all know that DBCA first prepares an instance and do a clone. So it’s trying to create the instance as per the plan but failing to start the instance saying the instance is already exist!!!

I swear I’ve removed each and every file related to Oracle even the registry entries :)
It’s a clean machine with just Oracle installed but why this thing is happening???

See the log below to get a clarity on the issue.

Copying database files
DBCA_PROGRESS : 1%
DBCA_PROGRESS : 2%
ORA-01075: you are currently logged on
ORA-01081: cannot start already-running ORACLE – shut it down first
DBCA_PROGRESS : 4%
DBCA_PROGRESS : 37%
Creating and starting Oracle instance
ORA-01075: you are currently logged on
DBCA_PROGRESS : 38%
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-01092: ORACLE instance terminated. Disconnection forced
DBCA_PROGRESS : 100%

After digging in, and doing a fair amount of research, I found the mistake that I’ve committed!!!

I’ve cleaned the physical files but what about the memory structure Oracle built in the operating system’s shared memory segments?

Yes, your guess is correct; The memory structures created by Oracle are remained intact in the shared memory. An ipcs has shown me that there are the latches and semaphores acquired by this particular Oracle instance were not released.

I wouldn’t have come across this issue if I’ve chosen a different name for my instance or with the same name but from different ORACLE_HOME as Oracle’s key to acquire the shared memory structures is the combination of OH and SID. If I use a different OH or SID, the DBCA will try to acquire the latches and the semaphores using this new key.

So I did a shutdown abort (I could do any kind of shutdown in fact but my idea is to do it as fast as possible) before doing the brute-force reset, which would release all the shared memory structures, latches and semaphores cleanly and able to complete my task.

The biggest mystery is why DBCA is not aborting upon noticing this? why it’s reporting a success upon a ‘disastrous failure’??

Oracle confirmed it’s a bug in DBCA not to report failures in silent mode. So you couldn’t just rely on the return status to the OS but have to do a lot more validations to confirm your mission is accomplished :)

I welcome your feedback and suggestions.

Regards,
CSM