How to improve the efficiency of time.strptime in Python?

Vote count:

0

I use Spyder's profiler to run a python script, which handles 700000 lines of data, and the time.strptime function takes more than 60s(the built-in function sort only takes 11s).

How should I improve its efficiency? Is there any effective module for time manipulation?

The core code snippet is here:

data = []
fr = open('big_data_out.txt')
for line in fr.readlines():
    curLine = line.strip().split(',')
    curLine[2] = time.strptime( curLine[2], '%Y-%m-%d-%H:%M:%S')
    curLine[5] = time.strptime( curLine[5], '%Y-%m-%d-%H:%M:%S')
#    print curLine
    data.append(curLine)

data.sort(key = lambda l:( l[2], l[5], l[7]) )
#print data

result = []
for itm in data:
    if itm[2] >= start_time and itm[5] <= end_time and itm[1] == cameraID1 and itm[4] == cameraID2:
        result.append(itm)

asked 42 secs ago

babykiss4ever

3

How to improve the efficiency of time.strptime in Python?