Troubleshooting iRODS

FAQ

GENERAL iRODS

Is there a diagram somewhere of how this stuff is setup?

• For NCMIR with CCDB, there is a diagram, here.

What directories store iRODS "stuff"?

• /opt/irods
• /etc/irods
• /var/opt/irods
• /var/run/irods
• /etc/init.d
  . What files are most important in configuration and troubleshooting of iRODS?
• /opt/irods
  • /server/bin/irodsctl (binary invoked by initscript)
  • /server/config/reConfig (iRODS rules)
• /etc/irods
  • /<proj><env>irods (configuration files that tune the binary to a project and environment)
• /var/opt/irods
  • /<proj><env>irods/log (log files)
• /var/run/irods
  • /<proj><env>irods.pid (process lock file - sticks around if irods dies messily)

How can I increase the logging level?

• (iRODS 3.2) export spLogLevel=5, then restart iRODS
• (iRODS 3.2) spLogLevel can be set by editing irodsctl.pl or setting the environment variable spLogLevel and then restarting the server. If set to "5", it is equal to LOG_NOTICE which will log messages of the 'notice' severity and more severe (lower numbers).

What decides the default resource?

• (iRODS 2.5) This and other rules are in the /opt/irods/server/config/reConfig/*.irb files.
• (iRODS 3.2) This and other rules are in the /opt/irods/server/config/reConfig/*.re files.

Are the iRODS configuration files templated?

• Yes. See modules/m_irods/templates

Where do the values for the templates get set?

• (iRODS 2.5) In the project module. See modules/p_ccdb/manifests/init.pp for an example.
• (iRODS 3.2) In the m_irods module. See modules/m_irods/manifests/params.pp and modules/m_irods/manifests/ccdb/params.pp for an example.

How do I find all the files the iCAT thinks are on a particular resource?

• iquest "SELECT RESC_VAULT_PATH, DATA_PATH, DATA_NAME WHERE DATA_RESC_NAME = 'ucsd-ccdb-prod-nas5'"

How do I move data from one resource to another?

• In the example below:
  • permissions and ownership of the original data is changed
    • everything is owned by ccdbprodirods (aka rawdata)
    • ccdbprodirods (aka rawdata) has rw to everything
    • everything is group owned by gid=100 (aka neuro, aka users)
    • the group has read/write to anything in the montage processed_data reconstruction segmentation directories
  • any data already in iRODS is iphymv'd to the new resource, then
  • any files that have been generated, but not yet sync'd (i.e. cataloged in the iRODS database) are moved to the new resource in order to keep all the data associated with a given MPID together, then
  • any symbolic links that have been created are moved to the new resource

• [from the directory above the relevant project on the resource that has the data you need to move...]
  for i in project_20103/microscopy_46211
  do for j in montage processed_data reconstruction segmentation
  do  echo $i/$j
  chown -R 1138 $i
  chmod -R u+rw $i 
  chgrp -R 100 $i
  chmod -R g+rw $i/$j
  iphymv -r -M -R ucsd-ccdb-prod-nas2 /telescience/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/$i

  Before running the next bit, any directories that are missing in the new location need to be created

  cd $i
  find . -type f -exec mv -b --suffix=20110815 -u -v {} /ccdbprod/ccdbprod2/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/$i/{} \;
  find . -type l -exec mv -b --suffix=20110815 -u -v {} /ccdbprod/ccdbprod2/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/$i/{} \;
  done
  done

I'm getting "rcAuthResponse failed with error -827000 CAT_INVALID_USER" in the log. What's wrong?

• The user isn't in the iCAT. You may have misspelled the name, forgotten the extension, used the wrong zone or connected to the wrong environment.

I'm getting "rcAuthResponse failed with error -826000 CAT_INVALID_AUTHENTICATION" in the log. What's wrong?

• The password you entered is incorrect.

I'm getting "ERROR: rmUtil: rm error for /telescience/home/CCDB_DATA_USER.portal/irmCDUunreg, status = -528002 status = -528002 UNIX_FILE_RENAME_ERR, No such file or directory" What's wrong?

• Check the permissions on the disk. If you don't have permission to the unix file, it acts like the file isn't there.

I'm getting "iadmin rmresc ucsd-ccdb-prod-nas5 ; ERROR: Level 0: resource 'ucsd-ccdb-prod-nas5' contains one or more dataObjects ; ERROR: rcGeneralAdmin failed with error -835000 CAT_RESOURCE_NOT_EMPTY". What's wrong?

• Use the iquest command above for finding what the iCAT thinks is on a particular resource.

I'm getting "ERROR: rmUtil: rm error for /telescience/home/ccdbdevirods/irodsupgrade, status = -13000 status = -13000 SYS_NO_API_PRIV" when I try to 'irm' a file that I own. What's wrong?

• In iRODS 3.2, the ACL on the top level collection is restricted to the admin user only. Use the -f flag on the 'irm' command, or modify the ACL on the trash collection.

I'm getting "ERROR: connectToRhost: error returned from host ccdb-dev-icat32.crbs.ucsd.edu status = -38000 status = -38000 SYS_AGENT_INIT_ERR ; ERROR: _rcConnect: connectToRhost error, server on ccdb-dev-icat32.crbs.ucsd.edu is probably down status = -38000 SYS_AGENT_INIT_ERR" when I run 'ils' on the iCAT and it looks like the reagent is not starting because 'ps ax' has "10620 ? Ssl 0:00 ./irodsServer PROJECT=ccdbdevirods ; 10629 ? Z 0:00 [irodsReServer] <defunct>" What's wrong?

• The reAgent isn't starting. Check both the rodsLog and the reLog for info. It was a parsing problem with the ccdbdev.re file in this case.

" when I try to 'irm' a file that I own. What's wrong?

• In iRODS 3.2, the ACL on the top level collection is restricted to the admin user only. Use the -f flag on the 'irm' command, or modify the ACL on the trash collection.

Only rods user can write to dev1 and only rodsadmin can write to dev0. What's wrong?

• I moved the dev0 vault from the slave to the iCAT using the 'iadmin modresc' command. When I moved the vault back to the slave, the problem went away.

I'm trying to do Resource Monitoring. How the heck do you get this to work?

the rule is loaded in irods cron
```iqstat -v
id     name
15249471
    msiServerMonPerf("verbose","default");
```
I modified the path in the script because it traversed up one dir too far
```#$path = dirname($0)."/../../config/scriptMonPerf.config";
$path = dirname($0)."/../config/scriptMonPerf.config";
```
when I run the script by hand it runs and generates the scriptMonPerf.config file but gives an error
``` /var/lib/irods/msiExecCmd_bin/irodsServerMonPerf -v
/var/lib/irods/msiExecCmd_bin/irodsServerMonPerf: File system path is missing!
#0#96#98#0##0#0##
=========== Server Name : nyan.crbs.ucsd.edu ===========
=Msc= Operating system => Linux
=Msc= Number of processors => 8
=CPU= CPU utilization rate (%) => 0
=CPU= Run queue load => 0
=Mem= Total memory (Bytes) => 16823705600
=Mem= Fraction of the memory being used (%) => 96
=Mem= Fraction of the swap memory being used (%) => 98
=Mem= Pagging I/O activity => 0
=Dsk= Disk occupation ratio (%) for "" =>
=Dsk= Disk volume available (MB) for "" =>
=Net= Network interface max capacity (Mbits/s) => 1000
=Net= Network input activity (Mbits/s) => 1.368
=Net= Network output activity (Mbits/s) => 0.2736
=Net= Network input activity rate (%) => 0
=Net= Network ouput activity rate (%) => 0
```
I'm guessing that the fs is passed by the rule and is currently "default" what should I set it to? or does it come from somewhere else?

[11:17] 
 ```-bash-4.2$ cat scriptMonPerf.config
systype:Linux
rhost:nyan.crbs.ucsd.edu
proc:8
swap:3
free:4
iswap:7
oswap:8
idle:15
mtu:1
iname:0
ipack:2
opack:6
diskused:0
diskabsavail:0```

• from https://groups.google.com/forum/#!topic/irod-chat/bGbVohu-ohw

> Hi Hao, 
> thx, this actually what i did. I realized that the core.re file doesn't accept the documented syntax: 
> { delay ...  } 
> input null 
> output ... 

you have to store this rule in a separate file on your local 
file system or on an iRODS resource. It will be kept in the iCAT 
as it is a delayed rule being run periodically. It is a little bit like a 
cron tab. 

What does the process list look like when I use scp to copy something to iRODS via fuse?

18380 pts/0    S+     0:18 scp -p -C -r crbs-operator@poas.crbs.ucsd.edu:/datapool/ccdb/datajail/ccdbproddj0/gatan/ebushong/CCDBID_68618 .
18381 pts/0    S+     3:16 /usr/bin/ssh -x -oForwardAgent no -oPermitLocalCommand no -oClearAllForwardings yes -C -lcrbs-operator poas.crbs.ucsd.edu scp -r -p -f /datapool/ccdb/datajail
18388 ?        S      0:00 irodsAgent
18389 ?        Ss     0:00 postgres: ccdbdevirods ccdbdevicat32 132.239.22.13(35895) idle  
18394 ?        S      0:03 irodsAgent
18395 ?        Ss     0:01 postgres: ccdbdevirods ccdbdevicat32 132.239.22.13(35898) idle  
18399 ?        S      0:03 irodsAgent
18400 ?        Ss     0:01 postgres: ccdbdevirods ccdbdevicat32 132.239.22.13(35901) idle  
18401 ?        S      0:00 [flush-253:2]
19430 ?        S      0:00 irodsAgent
19431 ?        Ss     0:00 postgres: ccdbdevirods ccdbdevicat32 132.239.22.13(36124) idle  
19434 ?        S      0:00 irodsAgent
19435 ?        Ss     0:00 postgres: ccdbdevirods ccdbdevicat32 132.239.22.13(36127) idle  
19436 ?        Z      0:00 [irodsAgent] <defunct>
19448 pts/1    R+     0:00 ps ax
24099 ?        Ssl    0:00 ./irodsServer PROJECT=ccdbdevirods
24108 ?        S      0:12 irodsReServer

 It looks like the iCAT can't talk to the Oracle database. What should I do?

If you see the following in the /var/opt/irods/ccdb<env>irods/log/rodslog... file:

Apr 27 13:19:29 pid:20623 NOTICE: environment variable set, irodsEnvFile=/var/opt/irods/ccdbdevirods/env
Apr 27 13:19:29 pid:20623 NOTICE: irodsUserName=ccdbdevirods
Apr 27 13:19:29 pid:20623 NOTICE: irodsPort=5630
Apr 27 13:19:29 pid:20623 NOTICE: irodsZone=telescience
Apr 27 13:19:29 pid:20623 NOTICE: irodsHost=armstrong.crbs.ucsd.edu
Apr 27 13:19:29 pid:20623 NOTICE: environment variable set, irodsPort=5630
Apr 27 13:19:29 pid:20623 NOTICE: environment variable set, irodsAuthFileName=/var/opt/irods/ccdbdevirods/auth
Apr 27 13:19:29 pid:20623 NOTICE: created irodsHome=/telescience/home/ccdbdevirods
Apr 27 13:19:29 pid:20623 NOTICE: created irodsCwd=/telescience/home/ccdbdevirods
    LocalHostName:  fugazi, fugazi.crbs.ucsd.edu, localhost, Port Num: 5630.

Apr 27 13:20:29 pid:20623 ERROR: cllConnect: OCILogon failed: -1
Apr 27 13:20:29 pid:20623 ERROR: OCI_Error: ORA-12170: TNS:Connect timeout occurred
Apr 27 13:20:29 pid:20623 NOTICE: chlOpen cmlOpen failure -803000
Apr 27 13:20:29 pid:20623 NOTICE: connectRcat: chlOpen Error. Status = -803000
Apr 27 13:20:29 pid:20623 SYSTEM FATAL: initServerInfo: no rcatHost error, status = -803000
Apr 27 13:20:29 pid:20623 NOTICE: initServer: initServerInfo error, status = -803000
Apr 27 13:20:29 pid:20623 ERROR: initServerMain: initServer error. status = -803000

Try this:

sqlplus ccdbdevicat@orcl1

or this

h5. Quick Database Connectivity Test
for i in ccdbdevicat ccdbdevmcat ccdbstageicat ccdbstagemcat crbsdevicat crbsdevmcat crbsstageicat crbsstagemcat ; do echo $i ; sqlplus $i@$i ; done

If that doesn't work, check the status of the database server, including checking disk space, load, and that the database engine is running.

CCDB Portal iRODS

I want to Archive data that isn't in the data jail. What should I do?

On the poas.crbs.ucsd.edu:

• mv the directory that contains the data to the appropriate place in the data jail and rename the directory to include the prefix "CCDBID_<MPID>_"

  • e.g.

      mv /export/crbs/ncmirdata5/eric/3view/Christine/ONH_506-L /export/crbs/ncmirdata5/rawdata/data/3view/CCDBID_45397_ONH_506-L
      mv /export/crbs/ncmirdata5/eric/3view/Christine/507-L /export/crbs/ncmirdata5/rawdata/data/3view/CCDBID_45385_507-L
      mv /export/crbs/ncmirdata5/eric/3view/Christine/507-R /export/crbs/ncmirdata5/rawdata/data/3view/CCDBID_45373_507-R
      mv /export/crbs/ncmirdata5/eric/3view/Branfman/Mouse81_cortex /export/crbs/ncmirdata5/rawdata/data/3view/CCDBID_43091_Mouse81_cortex

• recursively change the ownership of the data

  • e.g. chown -R rawdata:users CCDBID_<MPID>_<DATA_DESC>

      chown -R rawdata:users CCDBID_45397_ONH_506-L CCDBID_45373_507-R CCDBID_43091_Mouse81_cortex CCDBID_45385_507-L

The Archive process initiated from the CCDB Portal was interrupted. What should I do?

• FASTER OPTION - Use if no data made it to the final archive destination

  • Clear out the temporary storage space in the archive

    * irm -U -r the /telescience/home/CCDB_DATA_USER.portal/<scope>/<MPID_directory>

  • Remove any <MPID_directory> data from the archive

    Archive Rollback Cautions

    Only do this if you're SURE it's safe to delete everything under the <MPID_directory> (It should be either empty, or have only data for which a copy still exists in the data jail. The next command assumes all rawdata in the archive for that MPID can be DELETED. All rawdata for that <MPID_directory> must be in the data jail before using this procedure.

    rm -rf the /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_####/<MPID_directory>

  • "Archive" the data again from the CCDB Portal

ALTERNATIVE

• SAFER OPTION - Use if some, but not all data made it to the final archive destination

  • md5sum each file in the data jail and compare to md5sum for corresponding file in final archive destination

    cd /ccdbprod/ccdbproddj0/<scope> ; for i in `find <MPID_directory> -type f` ; do md5sum $i ; md5sum /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_####/$i/done

  • If the md5sums do not match, correct it so that the correct file is in both places.

  • Do an iscan on the archive directory to see if any files were registered with iRODS, but which never made it to the right place on the disk.

    If files show up as "orphaned" (meaning it's on the disk, but not cataloged in iRODs), they need to be "sync"d via the CCDB Portal.

    [ccdbprodirods@ccdb-irods rawdata]$ iscan -r /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249
    /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249/CCDB_MetaData/sutterlinGrid2Control_instrumentMetaData.txt tagged as orphan file

    An ils of an orphaned file will says it doesn't exist in iRODS.

    [ccdbprodirods@ccdb-irods rawdata]$ ils -L /telescience/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249/CCDB_MetaData/sutterlinGrid2Control_instrumentMetaData.txt
    ERROR: lsUtil: srcPath /telescience/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249/CCDB_MetaData/sutterlinGrid2Control_instrumentMetaData.txt does not exist or user lacks access permission

  • Copy any missing files from the data jail to the final archive location

    Copy Caution

    Be careful not to overwrite data you don't want to be overwritten - copy only the missing files, or try using 'cp -i ...'

    [ccdbprodirods@ccdb-irods rawdata]$ cp -r -i /ccdbprod/ccdbproddj0/3view/CCDBID_45249/<some_dir>/<some_file> /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249/<some_dir>/<some_file>
    

  • "sync" the archive directory via the CCDB Portal.
    The portal will send email telling you the data has been synced.

    From: ccdbuser@gmail.com
    To: vrowley@ncmir.ucsd.edu
    Message-ID: <1579866481.17.1302902712555.JavaMail.tomcat@melbourne.crbs.ucsd.edu>
    Subject: NCMIR data registration - synchronization
    MIME-Version: 1.0
    Content-Type: text/html; charset=us-ascii
    Content-Transfer-Encoding: 7bit
    
    Your data, /telescience/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249/CCDB_MetaData has been successfully synchronized with the NCMIR storage.
    /////////////////////Debugging//////////////////////////////
    .
    250 2.0.0 OK 1302902715 i2sm654494pbb.71
    QUIT
    221 2.0.0 closing connection i2sm654494pbb.71
    Finished Sending--------------------------------------

• • Rerun the iscan to verify it runs clean.

    [ccdbprodirods@ccdb-irods rawdata]$ iscan -r /ccdbprod/ccdbprod1/home/CCDB_DATA_USER.portal/CCDB_DATA_USER/acquisition/project_20099/microscopy_45249/rawdata/CCDBID_45249
    [ccdbprodirods@ccdb-irods rawdata]$ 

How do I know if the service started properly?

• ON AN ICAT you'll see both an irodsServer ANDand irodsReServer process running. The output of `ps ax | grep irod` should look like this:

  [root@fugazi ~]# ps ax | grep irod
  20609 ?        Ss     0:00 ./irodsServer PROJECT=ccdbdevirods
  20616 ?        S      0:00 irodsReServer
  22514 pts/3    R+     0:00 grep irod
  [root@fugazi ~]# 
  

• ON A SLAVE you'll see onlyan irodsServer process running. The output of `ps ax | grep irod` should look like this:

  [root@fugazi ~]# ps ax | grep irod
  20609 ?        Ss     0:00 ./irodsServer PROJECT=ccdbdevirods
  22514 pts/3    R+     0:00 grep irod
  [root@fugazi ~]# 
  

REGRESSION TESTING

DEV TEST

service ccdbdevirods stop ; /opt/irods/server/bin/irodsctl ccdbdevirods start ; tail -f /var/opt/irods/ccdbdevirods/log/rodsLog.2010.6.16 ; ps ax | grep irod

OUTPUT

Jun 20 19:52:54 pid:18609 NOTICE: environment variable set, irodsPort=5630
Jun 20 19:52:54 pid:18609 NOTICE: environment variable set, irodsAuthFileName=/var/opt/irods/ccdbdevirods/auth
Jun 20 19:52:54 pid:18609 NOTICE: created irodsHome=/telescience/home/ccdbdevirods
Jun 20 19:52:54 pid:18609 NOTICE: created irodsCwd=/telescience/home/ccdbdevirods
    LocalHostName:  fugazi, fugazi.crbs.ucsd.edu, localhost, Port Num: 5630.

Jun 20 19:52:54 pid:18609 ERROR: initZone: zoneName in env telescience does not match CCDBDevZone in icat 
Local Resource configuration: 
   RescName: myresc, VaultPath: /data1/testVault
   RescName: SRB-mcat-to-icat-dev, VaultPath: /data1/SRBVault/mcat-to-icat-Vault
   RescName: demoResc, VaultPath: /home/ccdbdevirods/iRODS/Vault
   RescName: SRB-mcat-to-icat-dev-nas0, VaultPath: /data1/SRBVault/mcat-to-icat-Vault
   RescName: ucsd-ccdb-dev-nas0, VaultPath: /nas/nas0/ccdbdevirods/iRODS/Vault

Zone Info:
    ZoneName: telescience   Type: LOCAL_ICAT    HostAddr: fugazi   PortNum: 5630

reHost:   fugazi

Jun 20 19:52:54 pid:18609 NOTICE: rodsServer Release version rods2.3 - API Version d is up
Jun 20 19:52:54 pid:18609 NOTICE: Starting irodsReServer

STAGE TEST

service ccdbstageirods stop ; /opt/irods/server/bin/irodsctl ccdbstageirods start ; tail -f /var/opt/irods/ccdbstageirods/log/rodsLog.2010.6.16 ; ps ax | grep irod

OUTPUT

Jun 20 19:48:56 pid:18497 NOTICE: environment variable set, irodsPort=5620
Jun 20 19:48:56 pid:18497 NOTICE: environment variable set, irodsAuthFileName=/var/opt/irods/ccdbstageirods/auth
Jun 20 19:48:56 pid:18497 NOTICE: created irodsHome=/telescience/home/ccdbstageirods
Jun 20 19:48:56 pid:18497 NOTICE: created irodsCwd=/telescience/home/ccdbstageirods
    LocalHostName:  crockett, crockett.crbs.ucsd.edu, localhost, Port Num: 5620.

Local Resource configuration: 
   RescName: SRB-z-ucsd-ccdb-nas1, VaultPath: /data1/telescience
   RescName: SRB-z-ucsd-ccdb-nas2, VaultPath: /data1/tele_home
   RescName: demoResc, VaultPath: /home/ccdbstageirods/iRODS/Vault
   RescName: SRB-z-ucsd-ccdb-nas0, VaultPath: /data1/SRBVault
   RescName: SRB-z-mcat-to-icat-nas0, VaultPath: /data1/mcat_icat
   RescName: SRB-testRes, VaultPath: /data1/SRBTestVault
   RescName: SRB-z-ucsd-ccdb-nas3, VaultPath: /data1/tele_old

Zone Info:
    ZoneName: telescience   Type: LOCAL_ICAT    HostAddr: crockett   PortNum: 5620

reHost:   crockett

Jun 20 19:48:56 pid:18497 NOTICE: rodsServer Release version rods2.3 - API Version d is up
Jun 20 19:48:56 pid:18497 NOTICE: Starting irodsReServer

PROD TEST

service ccdbprodirods stop ; /opt/irods/server/bin/irodsctl ccdbprodirods start ; tail -f /var/opt/irods/ccdbprodirods/log/rodsLog.2010.6.16 ; ps ax | grep irod

OUTPUT

Jun 20 19:51:07 pid:27192 NOTICE: irodsUserName=ccdbprodirods
Jun 20 19:51:07 pid:27192 NOTICE: irodsPort=5610
Jun 20 19:51:07 pid:27192 NOTICE: irodsZone=CCDBProdZone
Jun 20 19:51:07 pid:27192 NOTICE: irodsHost=localhost
Jun 20 19:51:07 pid:27192 NOTICE: environment variable set, irodsPort=5610
Jun 20 19:51:07 pid:27192 NOTICE: environment variable set, irodsAuthFileName=/var/opt/irods/ccdbprodirods/auth
Jun 20 19:51:07 pid:27192 NOTICE: created irodsHome=/CCDBProdZone/home/ccdbprodirods
Jun 20 19:51:07 pid:27192 NOTICE: created irodsCwd=/CCDBProdZone/home/ccdbprodirods
    LocalHostName:  lakota, lakota.crbs.ucsd.edu, localhost, Port Num: 5610.

Local Resource configuration: 

   No Local Resource Configured
Zone Info:
    ZoneName: CCDBProdZone   Type: LOCAL_ICAT    HostAddr: lakota   PortNum: 5610

reHost:   lakota

Jun 20 19:51:07 pid:27192 NOTICE: rodsServer Release version rods2.3 - API Version d is up
Jun 20 19:51:07 pid:27192 NOTICE: Starting irodsReServer

• fugazi, crocket and lakota are all icats
• irods logs are in /var/opt/irods/ccdb<env>irods/log/*
• the pid is in /var/run/ccdb<env>irods.pid
• the exectuables are in /opt/irods/
• the configuration files are in /etc/irods/ccdb<env>irods