~ / archivo / stackoverflow / 50337880
Combine lines 1-8, 9-16, etc. in a text file using Python
-
zemone pregunta 2018-05-14 19:42
I have a large text file that looks like this:
line1: Date line2: data line3: data line4: data line5: data line6: data line7: data line8: (blank) line9: Date (repeats itself from here)I need to read this txt file and merge lines1-8 onto a single line, save this line as a variable, then move on to lines9-16, merge onto a single line, save this line as a variable, etc. Any idea how to code this? Alternatively if there's a way to remove every '\n' except for every 8th '\n' that might even be easier, but I don't know how.
- zemone 2018-05-14 21:13 Thank you! All answers are fantastic solutions. Much appreciated
-
Look at
grouperin theitertoolsrecipes in the docs:from itertools import zip_longest def grouper(iterable, n, fillvalue=None): "Collect data into fixed-length chunks or blocks" # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx" args = [iter(iterable)] * n return zip_longest(*args, fillvalue=fillvalue)Now you can loop over every group of 8 lines:
with open(path) as f: for group in grouper(f, 8):If you want to join each group together into a single string that you can write to a new file:
with open(path) as f: for i, group in enumerate(grouper(f, 8)): with open(f'out_{i}.txt', 'w') as outf: outf.write(''.join(group))If you want to turn all the internal newlines into spaces:
with open(path) as f: for i, group in enumerate(grouper(f, 8)): with open(f'out_{i}.txt', 'w') as outf: outf.write(''.join(group).replace('\n', ' ') + '\n')The nice thing about doing things generator-style is that you can change your grouping rule without changing the rest of the code. For example, if you want to split on every line that matches a date format, as in Miguel Ortiz's answer:
from itertools import groupby rdate = re.compile(r'(\d+/\d+/\d+)') with open(path) as f: for i, (k, group) in enumerate(groupby(f, rdate.search)): with open(f'out_{i}.txt', 'w') as outf: outf.write(''.join(group).replace('\n', ' ') + '\n')- Miguel Ortiz yo 2018-05-14 20:50 I like your answer and how simple it's for a fixed input.
-
Split by "\n" then make a list of chunks of 8 lines of data joined via "," (or whatever delimiter you want):
with open('filename.txt', 'r') as f: l = f.read().split('\n') output = [','.join(l[j*8:j*8+8]) for j in range(int(len(l)/8))] -
I like the answers, but I add mine which has more control about the content of the file and could be used in the future if the amount of lines in the input file varies, (of course you can add more control to cover more corner cases).
This one will look for a line with date format, then associate the data to that date:
#!/usr/bin/python # parse zemone's data import re fileName="zemone.txt" data={} with open(fileName) as f: for line in f : if re.search(r'(\d+/\d+/\d+)',line) : myIndex=line.strip('\n') data[myIndex]='' elif line == '\n' : pass else : data[myIndex] += ' ' + line.strip('\n') for x in data : print x, data[x]Output:
mortiz@alberta:~/Documents/projects/python/data_manipulation$ python parse_zemone.py 01/01/2018 data1 data2 data3 data4 data5 data6 01/02/2018 ndata1 ndata2 ndata3 ndata4 ndata5 ndata6txt file
mortiz@alberta:~/Documents/projects/python/data_manipulation$ cat zemone.txt 01/01/2018 data1 data2 data3 data4 data5 data6 01/02/2018 ndata1 ndata2 ndata3 ndata4 ndata5 ndata6- abarnert 2018-05-14 20:55 Try out
for group in itertools.groupby(f, re.compile(r'(\d+/\d+/\d+)').search): - Miguel Ortiz yo 2018-05-14 21:27 Thanks abarnert! I'll look it tomorrow.
- abarnert 2018-05-14 20:55 Try out