Project

General

Profile

« Previous | Next » 

Revision 9514

Added by Aaron Marcuse-Kubitza over 11 years ago

bin/tnrs_db: Iterate over unscrubbed verbatim taxonlabels: put loop body in a function (which returns whether or not the loop should continue), so that the loop body can easily be wrapped in a transaction using sql.with_savepoint()

View differences:

tnrs_db
50 50
51 51 
    # Iterate over unscrubbed verbatim taxonlabels
52 52 
    total_pause = 0
53 
    while True:
53 
    def process():
54 54 
        # Fetch next set
55 55 
        cur = sql.select(db, tnrs_input, limit=tnrs.max_names, cacheable=False)
56 56 
        this_ct = cur.rowcount
57 57 
        log('Processing '+str(this_ct)+' taxonlabels')
58 58 
        if this_ct == 0:
59 
            if not wait: break
59 
            if not wait: return False
60 60 
            log('Waited '+str(total_pause)+' sec total')
61 61 
            total_pause += pause
62 
            if total_pause > max_pause: break
62 
            if total_pause > max_pause: return False
63 63 
            log('Waiting '+str(pause)+' sec...')
64 64 
            time.sleep(pause) # wait for more rows
65 
            continue # try again
65 
            return True # try again
66 66 
        # otherwise, rows found
67 67 
        total_pause = 0
68 68 
        names = list(sql.values(cur))
......
81 81 
        header.insert(0, time_col_name)
82 82 
        reader = csvs.ColInsertFilter(reader, now_str)
83 83 
        sql_io.append_csv(db, tnrs_data, reader, header)
84
85 
        return True # continue
86 
    while process(): pass
84 87 

  		




  85
  88
  
    
main()


  		












Also available in:  Unified diff