|
|
 |
Dev NotesThere are a few scripts that are needed to be run to upload the Affy arrays, update Affy annotation and to load Mas 5.0 expression data into the database. Some times things might not work because a data format has changed or a piece of data has not be entered. Below is a description of these scripts and the current location of the support files plus the location of the folders where the data is written out to
| Loading Affy Array Info | After the arrays are scanned and deposited on the filer, a script called "load_affy_array_files.pl" runs via cron job looking for new arrays.
| 1) | current location of the Script
| |
| /net/dblocal/www/html/sbeams/lib/scripts/Microarray
| | 2) |
| |
|
Current Command line options
# typical mode, adds any new files
1) ./load_affy_array_files.pl --run_mode add_new
# will parse the sample tag information and stomp the data in the database
2) ./load_affy_array_files.pl --run_mode update --method set_afs_sample_tag
# Delete the array with the file root given but LEAVES the sample
3) ./load_affy_array_files.pl --run_mode delete --array 20040609_02_LPS1-50
# removes both the array and sample records for the two root_file names
4) ./load_affy_array_files.pl --run_mode delete --both 20040609_02_LPS1-40 20040609_02_LPS1-50
| | 3) | Location of the error Log to see if anything bad happened
| |
|
/net/dblocal/www/html/sbeams/tmp/Microarray/AFFY_ERROR_LOG.txt
| | 4) | What to do if there is an error
| |
|
Run the script Manually and save the info the script will generate in debug mode
Run the script in debug mode and to see what the error is
./load_affy_array_files.pl --run_mode add_new --debug 1 --verbose 1> save_the_output.txt
| | 5) | Read the output to try and figure out what happened
| |
| 6) | Usually problems are generated from not first adding some support information into the database. Below are a few problems that have been seen
| |
|
Not adding a new project
Not adding a new user to both the Contact and user_login table
Not adding a new array type
|
|
| Add Affy Annotation Script | Every quarter Affymetrix releases a new annotation files that provides annotation for all the probe sets for the arrays they currently offer. This data is take in-house parsed and uploaded into SBEAMS to provide annotation for some of the Affy analysis pages. The Script "load_affy_annotation_files.pl" is used to do this job
| 1) | How to run the Affy annotation script
| |
| 2) | Location of the script
| |
|
/net/dblocal/www/html/sbeams/lib/scripts/Microarray
| | 3) | Command Line Options to run
| |
|
#Add new data to the database
1)./load_affy_annotation_files.pl --run_mode update --file_name full_path to annotation file
#Delete a file from the database
2)./load_affy_annotation_files.pl --run_mode delete --file_name full_path to annotation file
|
|
| R_CHP upload -- Add Mas 5.0 expression values | For each array all the expression values are loaded into the database after processing the arrays with the Mas5.0 algorithms. When the script was written there was no parsers for the *.CHP file generated by the Affy software so R-Bioconducor was used process the data. This script takes the CEL file reads it in, converts the data to Mas5.0 expression values and adds the data to the SBEAMS table. Script name load_affy_R_CHP_files.pl
| 1) | How to run the Affy R_CHP script
| |
| 2) | Location of the script
| |
|
/net/dblocal/www/html/sbeams/lib/scripts/Microarray
| | 3) | Command Line Options to run
| |
|
# typical mode, adds any new files
1) ./load_affy_R_CHP_files.pl --run_mode add_new
# Re upload the data and or re-compute the R run
2) ./load_affy_R_CHP_files.pl --run_mode update --redo_R yes or no
#give the affy_array_ids to update
3) ./load_affy_R_CHP_files.pl --run_mode update --redo_R yes or no --files 123,124,134
|
|
| Load External Data sets | Loading external dataset will utilize small INFO files, produced by Pre_process_affy_info_file.pl script. This script will parse a "master" info file which will contain all the sample and array annotation required to upload a file. The INFO file will be used in place of an XML file to upload a CEL file. Have the user fill out the spreadsheet.
| 1) | Add a new folder to "Affymetrix/core/probe_data/external"
| |
| 2) | Add all the CEL files to upload to the new folder
| |
| 3) | Fill out the annotation spreadsheet
| |
| 4) | Save the annotation spreadsheet as TAB delimited into the folder
| |
| 5) | Run the script
| |
|
./Pre_process_affy_info_file.pl --run_mode make_new \
--info_file give/path_to_file/info.txt
| | 6) | The script will do a crude check of the data and produce a INFO file for each CEL file
| |
| 7) | The script Loading Affy Arrays will then "notice" the files and upload them the next time the cron job launches the script
| |
|
|
