Saltar al contenido

~ / archivo / stackoverflow / 50337880

Combine lines 1-8, 9-16, etc. in a text file using Python

StackOverflow · python, python-2.7, fopen · ayudé
  1. zemone pregunta 2018-05-14 19:42

    I have a large text file that looks like this:

    line1: Date
    
    line2: data
    
    line3: data
    
    line4: data
    
    line5: data
    
    line6: data
    
    line7: data
    
    line8: (blank)
    
    line9: Date (repeats itself from here)
    

    I need to read this txt file and merge lines1-8 onto a single line, save this line as a variable, then move on to lines9-16, merge onto a single line, save this line as a variable, etc. Any idea how to code this? Alternatively if there's a way to remove every '\n' except for every 8th '\n' that might even be easier, but I don't know how.

    • zemone 2018-05-14 21:13 Thank you! All answers are fantastic solutions. Much appreciated
  2. abarnert respuesta +4 2018-05-14 19:57

    Look at grouper in the itertools recipes in the docs:

    from itertools import zip_longest
    
    def grouper(iterable, n, fillvalue=None):
        "Collect data into fixed-length chunks or blocks"
        # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
        args = [iter(iterable)] * n
        return zip_longest(*args, fillvalue=fillvalue)
    

    Now you can loop over every group of 8 lines:

    with open(path) as f:
        for group in grouper(f, 8):
    

    If you want to join each group together into a single string that you can write to a new file:

    with open(path) as f:
        for i, group in enumerate(grouper(f, 8)):
            with open(f'out_{i}.txt', 'w') as outf:
                outf.write(''.join(group))
    

    If you want to turn all the internal newlines into spaces:

    with open(path) as f:
        for i, group in enumerate(grouper(f, 8)):
            with open(f'out_{i}.txt', 'w') as outf:
                outf.write(''.join(group).replace('\n', ' ') + '\n')
    

    The nice thing about doing things generator-style is that you can change your grouping rule without changing the rest of the code. For example, if you want to split on every line that matches a date format, as in Miguel Ortiz's answer:

    from itertools import groupby
    
    rdate = re.compile(r'(\d+/\d+/\d+)')
    with open(path) as f:
        for i, (k, group) in enumerate(groupby(f, rdate.search)):
            with open(f'out_{i}.txt', 'w') as outf:
                outf.write(''.join(group).replace('\n', ' ') + '\n')
    
    • Miguel Ortiz yo 2018-05-14 20:50 I like your answer and how simple it's for a fixed input.
  3. mathisfun respuesta +2 2018-05-14 19:56

    Split by "\n" then make a list of chunks of 8 lines of data joined via "," (or whatever delimiter you want):

    with open('filename.txt', 'r') as f:
        l = f.read().split('\n')
    output = [','.join(l[j*8:j*8+8]) for j in range(int(len(l)/8))]
    
  4. Miguel Ortiz yo respuesta +2 2018-05-14 20:41

    I like the answers, but I add mine which has more control about the content of the file and could be used in the future if the amount of lines in the input file varies, (of course you can add more control to cover more corner cases).

    This one will look for a line with date format, then associate the data to that date:

    #!/usr/bin/python
    # parse zemone's data
    
    import re
    fileName="zemone.txt"
    data={}
    
    with open(fileName) as f:
        for line in f :
            if re.search(r'(\d+/\d+/\d+)',line) :
                myIndex=line.strip('\n')
                data[myIndex]=''
            elif line == '\n' :
                pass
            else :
                data[myIndex] += ' ' + line.strip('\n')
    
    for x in data :
        print x, data[x]
    

    Output:

    mortiz@alberta:~/Documents/projects/python/data_manipulation$ python parse_zemone.py 
    01/01/2018  data1 data2 data3 data4 data5 data6
    01/02/2018  ndata1 ndata2 ndata3 ndata4 ndata5 ndata6
    

    txt file

    mortiz@alberta:~/Documents/projects/python/data_manipulation$ cat zemone.txt 
    01/01/2018
    
    data1
    
    data2
    
    data3
    
    data4
    
    data5
    
    data6
    
    
    
    01/02/2018
    
    ndata1
    
    ndata2
    
    ndata3
    
    ndata4
    
    ndata5
    
    ndata6
    
    • abarnert 2018-05-14 20:55 Try out for group in itertools.groupby(f, re.compile(r'(\d+/\d+/\d+)').search):
    • Miguel Ortiz yo 2018-05-14 21:27 Thanks abarnert! I'll look it tomorrow.

← todos los hilos de StackOverflow