Project

General

Profile

« Previous | Next » 

Revision 9517

Added by Aaron Marcuse-Kubitza over 11 years ago

bin/tnrs_db: removed no longer needed explicit population of the Time_submitted, which is now done automatically by the tnrs table. however, this requires starting the transaction before submitting data, so Time_submitted is correctly set to the submission time rather than the insertion time. the setting of the correct time can be tested by inserting `time.sleep(n_sec)` after the TNRS request and checking that the Time_submitted is close to the time tnrs_db was run instead of n_sec seconds later.

View differences:

tnrs_db
27 27 

  		




  28
  28
  
    
tnrs_input = sql_gen.Table('tnrs_input_name')


  		




  29
  29
  
    
tnrs_data = sql_gen.Table('tnrs')


  		




  30
  
  
    
time_col_name = 'Time_submitted'


  		




  31
  30
  
    

  		




  32
  31
  
    
def main():


  		




  33
  32
  
    
    # Input


  		




  ......




  70
  69
  
    
        def process():


  		




  71
  70
  
    
            # Run TNRS


  		




  72
  71
  
    
            log('Making TNRS request')


  		




  73
  
  
    
            now_str = str(dates.now())


  		




  74
  72
  
    
            tnrs_profiler.start()


  		




  75
  73
  
    
            try: stream = tnrs.repeated_tnrs_request(names)


  		




  76
  74
  
    
            finally:


  		




  ......




  78
  76
  
    
                log('Cumulatively: '+tnrs_profiler.msg())


  		




  79
  77
  
    
            


  		




  80
  78
  
    
            log('Storing TNRS response data')


  		




  81
  
  
    
            reader, header = csvs.reader_and_header(stream)


  		




  82
  
  
    
            header.insert(0, time_col_name)


  		




  83
  
  
    
            reader = csvs.ColInsertFilter(reader, now_str)


  		




  84
  
  
    
            sql_io.append_csv(db, tnrs_data, reader, header)


  		




  
  79
  
    
            sql_io.append_csv(db, tnrs_data, *csvs.reader_and_header(stream))


  		




  85
  80
  
    
        # start transaction *before* submitting data, so Time_submitted is


  		




  86
  81
  
    
        # correctly set to the submission time rather than the insertion time.


  		




  87
  82
  
    
        # these may differ by several minutes if TNRS is slow.


  		












Also available in:  Unified diff