Scenario
While uploading the RPD the server was not restarting ,clicking on ‘coreapplication’on the EM was giving an error regarding ADF . To resolve this issue we had to restart the whole box after which the really issue started , the weblogic and opmnctl were not coming up.
Reason
The disk space on the server was full which was the reason for the servers not to come up. Due to restarting the server when the disk space was full corrupted all the files. The only way to bring the services up were to delete/replace all the corrupted files for all the services.
Solution
Part – I – Resolving the weblogic issue
In order for the weblogic to come up these are the following steps that were taken
1. CWallet.sso was deleted
The CWallet.sso file which has all the encrypted data is to be deleted or taken a backup on the following location mentioned in the screenshot.
2. Replicas.prop file has to be deleted.
The file replica.prop has to be deleted which is in the following location
$DOMAIN/servers/AdminServer/data/ldap/conf/replicas.prop
This file was null when we had this issue. Once this file was deleted and the weblogic.sh was executed the file had an entry.
Make sure that there is this entry in the file
replica.num=0
3. Changing the username and password in the boot properties.
Sometimes we would have to reset the username and password of weblogic in the boot properties file. Traverse to the following location
$DOMIAN\bifoundation_domain\servers\AdminServer\security
In the boot properties file edit the username and password and once the services are running it will get encrypted back.
Once these 3 steps were performed the weblogic was up and running and the status changed from FORCE SHUT DOWN.
Part – II – Resolving the opmnctl services issues
After the weblogic was up and running , as usual we tried to start the opmnctl services and guess what none of the services were getting started. It was showing all sorts of errors while running. For this we created a system.user credential key in em
- Following a Oracle Forum we created a system users in em under the following location Weblogic –>Expand –>bifoundation_domain –>Right Click –> Security –> Credential
- Create a map oracle.bi.system and a key system.user
- Create a credential key scheduler.schema with the information provided below
- Create a map oracle.biapps.system and a key (Not required but do it if the services do not come up)
2.While adding the first credential key the BI Server and the BI Presentation Services were up and running and the while doing the second step the Cluster and Schdeuler were up and running.
Different errors which were faced
- In the saw log in Presentation Services
State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred.
[nQSError: 12002] Socket communication error at call=Connect: (Number=107) Transport endpoint is not connected
[nQSError: 12010] Communication error connecting to remote end point: address = obalcdcqvm001; port = 9706.
[nQSError: 12008] Unable to connect to port 9706 on machine obalcdcqvm001. (HY000)[[
- In the Cluster logs
SUGEnterpriseCredentialRepository not initialized
[2015-08-26T06:18:37.000-04:00] [OracleBIClusterControllerComponent] [ERROR:1] [] [] [ecid: ] [tid: c65c4720] [71002] The Cluster Controller is terminating due to errors in the cluster configuration.
SUGEnterpriseCredentialRepository not initialized
Please note that when this issue was raised with Oracle the above mentioned error shows up when there was an error if there was a disk space issue .
Oracle SR Information – 3-11255809271