Notes installing the SBEAMS - Immunostain module

$Id: Immunostain.installnotes,v 1.0 2005/01/04


-------------------------------------------------------------------------------
1) Software Dependencies

You must first install the SBEAMS Core.  See the separate installation
notes (sbeams.installnotes) on how to accomplish that.

The following Perl Modules often not found on a standard UNIX/Linux
setup are required to successfully use SBEAMS - Immunostain (in addition
to the dependencies for the SBEAMS Core).

GD::GRAPH

2) Installation Location

SBEAMS is designed to live entirely in the "htdocs" area of your Apache
web server.  For the remainder of this installation, it will be assumed
that your installation is configured as follows; compensate for your
specific setup:
  servername: db
  DocumentRoot: /local/www/html
  Primary location: directly located in DocumentRoot,
                   /local/www/html/sbeams  --> http:/db/sbeams/
  Development location: In a dev1 tree starting in the DocumentRoot,
                   /local/www/html/dev1/sbeams  -> http:/db/dev1/sbeams/

All modules live in the same area and should be unpacked into the
main SBEAMS area.


-------------------------------------------------------------------------------
3) Create and populate the database

It is assumed that you have already created and tested your SBEAMS Core
database.  You may either create a separate database for the Immunostain 
or you can put everything in the same database.

Note that some database engines (rare now) may not permit
cross-database queries in which case your may NOT use separate
databases.  If you do use separate databases, you may not be able to
enforce referencial integrity between tables in the different
databases.  This may or may not be a significant concern.

- If you decide on a separate database, create it and within it,
  create users "sbeams" and "sbeamsro" as a read/write
  account and a read-only account, respectively, as done for the Core.

- Generate the appropriate schema for your type(s) of database as follows:

setenv SBEAMS /local/www/html/dev1/sbeams
cd $SBEAMS/lib/scripts/Core

foreach dbtype ( mssql mysql pgsql oracle )
  ./generate_schema.pl \
    --table_prop ../../conf/Immunostain/Immunostain_table_property.txt \
    --table_col ../../conf/Immunostain/Immunostain_table_column.txt \
    --schema_file ../../sql/Immunostain/Immunostain\
    
    --destination_type $dbtype
end


- Verify that the SQL CREATE and DROP statements have been correctly
  generated in $SBEAMS/lib/sql/Immunostain/

- Execute the statements to create and populate the database with some
  bare bones data and indexes (for faster loading and querying):

SQL Server Example:

To CREATE and POPULATE:
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_CREATETABLES.mssql -D sbeamsdev
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_POPULATE.mssql -D sbeamsdev
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_CREATECONSTRAINTS.mssql -D sbeamsdev

To CREATE indexes:
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_CREATEINDEXES.mssql -D sbeamsdev

To DROP:
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_DROPCONSTRAINTS.mssql -D sbeamsdev
sqsh -i $SBEAMS/lib/sql/Immunostain/Immunostain_DROPTABLES.mssql -D sbeamsdev

Note that the Immunostain_POPULATE.mssql is not autogenerated and should
probably work for all flavors of database

Examples for table creation for other database flavors can be found in the
Core installation notes and will not be repeated here.


-------------------------------------------------------------------------------
4) Edit the SBEAMS Configuration files

setenv SBEAMS /local/www/html/dev1/sbeams

cd $SBEAMS/lib/conf
edit SBEAMS.conf

Specifically:
set  
DBPREFIX{Immunostain}    = Immunostain.dbo. (if you created a new Immunostain database)
or 
DBPREFIX{Immunostain}    = "core database prefix"  (if you added the Immunostain tables to your core installed database))

-------------------------------------------------------------------------------
5) Populate the driver tables

cd $SBEAMS/lib/scripts/Core
set CONFDIR =  "../../conf"
./update_driver_tables.pl $CONFDIR/Immunostain/Immunostain_table_property.txt
./update_driver_tables.pl $CONFDIR/Immunostain/Immunostain_table_column.txt

If this doesn't work.  Do not proceed, debug first.


-------------------------------------------------------------------------------
6) Add groups

Log in via the web interface as a user with Administrator privileges,
switch to the Admin group using the pull-down menu at the top, and add
two work groups:
[SBEAMS Home] [Admin] [Manage Work Groups] [Add Work Group]
Add entries for (exacly as shown!):
  Immunostain_user
  Immunostain_admin
  Immunostain_readonly
(Note that after INSERTing the first, you can click [Back], edit the previous
information slightly, and click [INSERT] to add another.)

Immunostain_admin has privilege over all tables in the Immunostain
module, while the Immunostain_user only has access to certain tables
and may often not modify other users records.  The Immunostain_readonly
group is a separate group for looky loos.

Now go to [Admin, Manage User Group Associations], [Add ...], and add
yourself and whoever else to these groups as appropriate.

Now set up the table group securities:
  rowprivate - Immunostain_user - data_writer
  rowprivate - Immunostain_admin - data_writer
  rowprivate - Immunostain_readonly - data_writer
  project - Immunostain_user - data_writer
  project - Immunostain_admin - data_modifier
  Immunostain_infrastructure - Immunostain_admin - data_modifier
  Immunostain_infrastructure - Immunostain_user - data_writer



Now that the Immunostain driver tables are loaded, and the groups have been
established, you should be able to go to the web site again and click on
SBEAMS - Immunostain and explore the tables.  They're all going to be empty,
but you shouldn't get any errors, just empty resultsets.

If this doesn't work.  Do not proceed, debug first.


-------------------------------------------------------------------------------
7) Add some sample data

(logged in as your user account)

Add a Project as follows:

- Switch to the Immunostain_user group by using the drop-down box at top
- Click on [Immunostain Home]
- Under "Projects You Own", click [Add A New Project].  Required
  fields are in red.  If you don't have a budget number, enter NA.
- Fill in the appropriate information and [INSERT]

Note: you may want to go to http://www.scgap.org/, click on Prostate/Bladder cell GAP to browse our 
immunostain for data examples. It also has great examples and explainations of terms, structural Units,
bulk loading examples, etc.
Also there are 4 data examples (humanBladder.txt, humanProstate.txt, mouseBladder.txt, mouseProstate.txt)  you can try out at 
$SBEAMS/doc/Immunostain/

-Add data to
     Antigen,
     Antibodies,
    Probes (optional, this is important for insitu hybridization experiments)
    Cell Types 
    TissueType  (kidney, liver, prostate etc.)
    Detection Methods  (optional)
    Image Type
    Expression Level 
    Abundance
    Surgical Procedure (optional)
    Clinical Diagnosis  (optional)
    Protocols (optional, this table is part of the core installation)
    Organism (this table is part of the core installation. It is essential for the programmatic bulk loading of Immunostain samples)
    
    by clicking on the approbriate table on the left side
    Example: Cell Types --> Add Cell Type --> [INSERT])

-Add some sample  Immuostain as follows:
  we have developed a nomenclature for our specimens and subsequent sub classification
  have a look at $SBEAMS/doc/Immunostain/FileNameConvention.php   
- loading data can be done on the command line (more below)  or through the web interface
   Web Interface 
-Add a Specimen
-Add a SpecimenBlock
-Add an Assay (a stain or a slide)
-Add an AssayChannel (the default is channel1, channel would be a wavelength or a filter apllied when 
      viewing the slide in order to see a probe)
-Add a Slide Image (again refer to FileNameConvention.php for details on how to name them)
-Add Assay Image subfield if approbriate       
-Add a Characterization for each celltype (Structural Unit), abundance and Intensity level you specified above .  

Programmatically
use $SBEAMS/lib/scripts/Immunostain/ImmunoLoaderNew for uploading bulk data. 
Note: You will need to adjust the .conf Files in $SBEAMS/lib/scripts/Immunostain/ to specify your own tissue_type_name, organism
            protocol and project_id    
            we have four conf files listed (mouse/human, prostate/bladder)
            we are using formatted tab delimeted txt file to load data.
            go to http://www.scgap.org/, click on Prostate/Bladder cell GAP to download an xls template and an example.
            each combination of  organism-tissueType has its own unique xls template
            you need to edit, or delete and create,  the arrays at the very start of the ImmunoLoaderNew.pl since they reflect the 
            columns of the tab delimeted txt file to be loaded and are only useful for the mouse and human prostate/bladder tissue. 
            Furthermore, every instance of a column name (ex: 'lobe of prostate(central)') in the script should be changed with the column 
            you choose to replace it with. This may take some code changes depending on your column specifications in the tab delimited txt file

The loader will check for previously entered information about a particular specimen and update or insert new data 
based on the specimen. 
It also writes simple error information (such as Anitbody not found in database) to an error log file. The location of the error log 
is part of the command line option. 

example: 
    perl ./ImmunoLoaderNew.pl --tissue_type  xxx -- organism yyy  --source_file ttt (ttab delimited) 
    --error_file error.log

    tissue_type and organism should be the same as the name of the conf file
          



#################################################################################################################
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Troubleshooting

1) How to re-generate and update table schemas.

- Drop table and (firstly) its contraints, via SQL commands. Look under 
  DOMAIN_DROPCONSTRAINTS.mssql and DOMAIN_DROPTABLES.mssql.
  e.g. look under Core_DROPCONSTRAINTS.mssql and Core_DROPTABLES.mssql

- Edit the appropriate $DOMAIN_table_column.txt file
  e.g. make a field in conf/Core/Core_table_property.txt nullable=N

- Generate new schema files using generate_schema.pl
  e.g.  cd $SBEAMS/lib/scripts/Core
	./generate_schema.pl --table_prop ../../conf/Core/Core_table_property.txt --table_col ../../conf/Core/Core_table_column.txt --schema_file ../../sql/Core/Core --destination_type mssql

- Now re-create table and its contraints using the new (updated) dll.
  e.g. look under Core_CREATETABLES.mssql and Core_CREATECONSTRAINTS.mssql

- Populate the table, if required.
  e.g. look in Core_POPULATE.mssql




