| View previous topic :: View next topic |
| Author |
Message |
freke
l33t
Joined: 23 Jan 2003
Posts: 980
Location: Somewhere in Denmark
|
 Posted: Tue Jul 05, 2016 2:42 pm Post subject: Fileprocessing using awk/sed |
 |
|
Hi,
I have a 'comma' seperated file like this: (snippet) | Code: | "VOGN" "345" "Sydvestlolland"
"RUTE" "345" 1000 "FOR4931" "360 SV i 4930" "John (Jette Clausen)" "Sydvestlolland " "Forhandler" 24
"RUTE" "345" 1000 "FOR4984" "345 SV i 4983" "John (Jette Clausen)" "Sydvestlolland " "Forhandler" 5
"RUTE" "345" 1010 "30150" "Løjet" "Jørgen Christensen" "Hj. Højrebyvej og Løjetvej " "Almindelig" 2
"RUTE" "345" 1030 "71400" "Kassen Langholmvej" "Service leveringer" "TVÆRGADE 20 NYKØBING F." "Almindelig" 6
"RUTE" "345" 1050 "14200" "Arninge" "Traudi Gahleitner" "Tillitsevej 31 4983 DANNEMARE" "Almindelig" 16
"RUTE" "345" 2011 "26075" "Søllested 03" "Mattias Johansen" "Hovedgaden 26 4900 NAKSKOV" "Almindelig" 32
"RUTE" "345" 2020 "29300" "Langø" "Irene Kristiansen" "Kappelvej 3 " "Almindelig" 40
"RUTE" "345" 2020 "26100" "Søllested 01" "Mathias Gotfredsen" "Hovedgaden 26 4920 SØLLESTED" "Almindelig" 40
"RUTE" "345" 2021 "30100" "Vestenskov" "Dennis Frost" "Vestenskovvej 15 " "Almindelig" 15
"RUTE" "345" 2030 "26050" "Søllested 02" "Kevin Hansson" "Hovedgaden 26 4920 SØLLESTED" "Almindelig" 35
"RUTE" "345" 2040 "52300" "Skovlænge" "Hennig Møller Andersen" "Ullerslevvej og Nørredalsvej 4920 SØLLESTED" "Almindelig" 26
"RUTE" "345" 2060 "14400" "Avnede" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 15
"RUTE" "345" 2061 "22900" "Gurreby" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 19
"RUTE" "345" 2062 "58600" "Ullerslev 1" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 16
"RUTE" "345" 2063 "58700" "Torpe" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 18
"RUTE" "345" 2064 "23900" "Hellinge" "Hennig Møller Andersen" "Hjørnet Nørredalsvej og Ullerslevvej " "Almindelig" 24
"RUTE" "345" 2070 "44600" "Næsby Strand" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 32
"RUTE" "345" 2072 "59900" "Tillitse Fæland" "Traudi Gahleitner" "Tillitsevej 31 4900 NAKSKOV" "Almindelig" 12
"RUTE" "345" 2120 "17800" "Dannemare By" "Henriette Eriksen" "Dannemare Brugs " "Almindelig" 65
"RUTE" "345" 2121 "22300" "Græshave" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 16
"RUTE" "345" 2122 "62700" "Øster Skovby" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 9
"RUTE" "345" 2123 "56600" "Tillitse" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 18
"RUTE" "345" 2127 "25700" "Gloslunde - Hoby" "Jesper Eriksen" "Dannemre Brugs " "Almindelig" 22
"RUTE" "345" 2170 "30600" "Majbølle v/Tillitse" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 31
"TILGANG" "345" "30600" "5018994" "Leif Andersen" "MENSALVEJ 48 " "4983 DANNEMARE" "MA-TI-ON-TO-FR-LØ" ""
"RUTE" "345" 2171 "41300" "Munkeby" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 15
"RUTE" "345" 2180 "44500" "Næsby" "Bente Lillian Hansen" "Rudbjergvej 72 " "Almindelig" 12
"RUTE" "345" 2190 "51200" "Sjunkeby" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 12
"RUTE" "345" 2191 "27800" "Knubbeløkke" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 9
"RUTE" "345" 2192 "26400" "Vesternæs" "Jesper Eriksen" "Dannemare Brugs 4983 DANNEMARE" "Almindelig" 9 |
Which I need slightly modified, I want this output: | Code: | "RUTE" "345 Sydvestlolland" 1000 "FOR4931" "360 SV i 4930" "John (Jette Clausen)" "Sydvestlolland " "Forhandler" 24
"RUTE" "345 Sydvestlolland" 1000 "FOR4984" "345 SV i 4983" "John (Jette Clausen)" "Sydvestlolland " "Forhandler" 5
"RUTE" "345 Sydvestlolland" 1010 "30150" "Løjet" "Jørgen Christensen" "Hj. Højrebyvej og Løjetvej " "Almindelig" 2
"RUTE" "345 Sydvestlolland" 1030 "71400" "Kassen Langholmvej" "Service leveringer" "TVÆRGADE 20 NYKØBING F." "Almindelig" 6
"RUTE" "345 Sydvestlolland" 1050 "14200" "Arninge" "Traudi Gahleitner" "Tillitsevej 31 4983 DANNEMARE" "Almindelig" 16
"RUTE" "345 Sydvestlolland" 2011 "26075" "Søllested 03" "Mattias Johansen" "Hovedgaden 26 4900 NAKSKOV" "Almindelig" 32
"RUTE" "345 Sydvestlolland" 2020 "29300" "Langø" "Irene Kristiansen" "Kappelvej 3 " "Almindelig" 40
"RUTE" "345 Sydvestlolland" 2020 "26100" "Søllested 01" "Mathias Gotfredsen" "Hovedgaden 26 4920 SØLLESTED" "Almindelig" 40
"RUTE" "345 Sydvestlolland" 2021 "30100" "Vestenskov" "Dennis Frost" "Vestenskovvej 15 " "Almindelig" 15
"RUTE" "345 Sydvestlolland" 2030 "26050" "Søllested 02" "Kevin Hansson" "Hovedgaden 26 4920 SØLLESTED" "Almindelig" 35
"RUTE" "345 Sydvestlolland" 2040 "52300" "Skovlænge" "Hennig Møller Andersen" "Ullerslevvej og Nørredalsvej 4920 SØLLESTED" "Almindelig" 26
"RUTE" "345 Sydvestlolland" 2060 "14400" "Avnede" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 15
"RUTE" "345 Sydvestlolland" 2061 "22900" "Gurreby" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 19
"RUTE" "345 Sydvestlolland" 2062 "58600" "Ullerslev 1" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 16
"RUTE" "345 Sydvestlolland" 2063 "58700" "Torpe" "Hennig Møller Andersen" "Hjørnet af Ullerslevvej og Nørredalsvej " "Almindelig" 18
"RUTE" "345 Sydvestlolland" 2064 "23900" "Hellinge" "Hennig Møller Andersen" "Hjørnet Nørredalsvej og Ullerslevvej " "Almindelig" 24
"RUTE" "345 Sydvestlolland" 2070 "44600" "Næsby Strand" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 32
"RUTE" "345 Sydvestlolland" 2072 "59900" "Tillitse Fæland" "Traudi Gahleitner" "Tillitsevej 31 4900 NAKSKOV" "Almindelig" 12
"RUTE" "345 Sydvestlolland" 2120 "17800" "Dannemare By" "Henriette Eriksen" "Dannemare Brugs " "Almindelig" 65
"RUTE" "345 Sydvestlolland" 2121 "22300" "Græshave" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 16
"RUTE" "345 Sydvestlolland" 2122 "62700" "Øster Skovby" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 9
"RUTE" "345 Sydvestlolland" 2123 "56600" "Tillitse" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 18
"RUTE" "345 Sydvestlolland" 2127 "25700" "Gloslunde - Hoby" "Jesper Eriksen" "Dannemre Brugs " "Almindelig" 22
"RUTE" "345 Sydvestlolland" 2170 "30600" "Majbølle v/Tillitse" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 31 "TILGANG" "345 Sydvestlolland" "30600" "5018994" "Leif Andersen" "MENSALVEJ 48 " "4983 DANNEMARE" "MA-TI-ON-TO-FR-LØ" ""
"RUTE" "345 Sydvestlolland" 2171 "41300" "Munkeby" "Traudi Gahleitner" "Tillitsevej 31 " "Almindelig" 15
"RUTE" "345 Sydvestlolland" 2180 "44500" "Næsby" "Bente Lillian Hansen" "Rudbjergvej 72 " "Almindelig" 12
"RUTE" "345 Sydvestlolland" 2190 "51200" "Sjunkeby" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 12
"RUTE" "345 Sydvestlolland" 2191 "27800" "Knubbeløkke" "Jesper Eriksen" "Dannemare Brugs " "Almindelig" 9
"RUTE" "345 Sydvestlolland" 2192 "26400" "Vesternæs" "Jesper Eriksen" "Dannemare Brugs 4983 DANNEMARE" "Almindelig" 9 |
ie. Every occurance of "345" is replaced by "345 Sydvestlolland" - would be star if sed/awk/whatever could get that from line 1 in the snippet.
Next lines containing "TILGANG" (or "AFGANG") in the first field, should be appended to the previous line - there might be multiple TILGANG/AFGANG in succession ie. | Code: | "VOGN" "361" "Maribo Redaktion"
"RUTE" "361" 3120 "40800" "Maribo 12" "Amir Seric" "BANEGÅRDSPLADSEN 2 4930 MARIBO" "Almindelig" 59
"TILGANG" "361" "40800" "5270835" "Jan Rasmussen" "MARGRETECENTERET 104 " "4930 MARIBO" "MA-TI-ON-TO-FR-LØ" ""
"AFGANG" "361" "40800" "5270835" "Jan Rasmussen" "ØSTRE BOULEVARD 12 104 " "4930 MARIBO" "MA-TI-ON-TO-FR-LØ" "" |
needs to end up as one line | Code: | | "RUTE" "361 Maribo Redaktion" 3120 "40800" "Maribo 12" "Amir Seric" "BANEGÅRDSPLADSEN 2 4930 MARIBO" "Almindelig" 59 "TILGANG" "361 Maribo Redaktion" "40800" "5270835" "Jan Rasmussen" "MARGRETECENTERET 104 " "4930 MARIBO" "MA-TI-ON-TO-FR-LØ" "" "AFGANG" "361 Maribo Redaktion" "40800" "5270835" "Jan Rasmussen" "ØSTRE BOULEVARD 12 104 " "4930 MARIBO" "MA-TI-ON-TO-FR-LØ" "" |
Lastly I need to remove any lines with 1-14 in the 9th field.
Achievable?
Any help greatly appriciated 
link to complete file: https://bpaste.net/show/67ce201d9346
[EDIT]Sorry - wrong section... MODS feel free to move  |
|
| Back to top |
|
 |
freke
l33t
Joined: 23 Jan 2003
Posts: 980
Location: Somewhere in Denmark
|
| Posted: Tue Jul 05, 2016 4:34 pm Post subject: |
|
|
I have this 'working' I think - double checking files now | Code: | sed -i.bak 's/\"100\"/\"100 Afkast og Kiosk\"/' test.txt
sed -i.bak 's/\"390\"/\"390 Sydfalster\"/' test.txt
sed -i.bak 's/\"370\"/\"370 Nysted\"/' test.txt
sed -i.bak 's/\"380\"/\"380 Nordfalster\"/' test.txt
sed -i.bak 's/\"335\"/\"335 Nakskov bude\"/' test.txt
sed -i.bak 's/\"330\"/\"330 Nordvestlolland\"/' test.txt
sed -i.bak 's/\"316\"/\"316 Nakskov Kiosk 1\"/' test.txt
sed -i.bak 's/\"345\"/\"345 Sydvestlolland\"/' test.txt
sed -i.bak 's/\"355\"/\"335 Nordlolland\"/' test.txt
sed -i.bak 's/\"361\"/\"361 Maribo Redaktion\"/' test.txt
sed -i.bak 's/\"350\"/\"350 Sydlolland\"/' test.txt
sed -i.bak 's/\"360\"/\"360 Maribo - Rødby\"/' test.txt
sed -i.bak 's/\"352\"/\"352 Sakskøbing depot\"'/ test.txt
sed -i.bak 's/\"385\"/\"385 Midtfalster\"/' test.txt
sed -i.bak 's/\"365\"/\"365 Østfalster\"/' test.txt
sed -i.bak 's/\"347\"/\"347 Guldborg\"/' test.txt
sed -i.bak -e :a -e '$!N;s/\n"AFGANG"/\t"AFGANG"/;ta' -e 'P;D' test.txt
sed -i.bak -e :a -e '$!N;s/\n"TILGANG"/\t"TILGANG"/;ta' -e 'P;D' test.txt
awk -F $'\t' '$9>14 {print $0}' test.txt > output.txt |
I'd like it to be able to figure out the number-mapping itself though - ie. if name/number changes. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21722
|
| Posted: Wed Jul 06, 2016 1:18 am Post subject: |
|
|
| If you do not need those backups, you can combine most of those calls. At a minimum, all of your number -> number+name transforms can be done in one step with sed -e 's/\"100\"/\"100 Afkast og Kiosk\"/' -e 's/\"390\"/\"390 Sydfalster\"/' etc. test.txt. Similarly, you could chain your invocations instead of repeatedly editing the file in place: sed transforms test.txt | awk transforms > output.txt. |
|
| Back to top |
|
|
|
Other Things Gentoo
