[an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] (none) [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] (none) [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive]
 
[an error occurred while processing this directive] [an error occurred while processing this directive]
Skåne Sjælland Linux User Group - http://www.sslug.dk Home   Subscribe   Mail Archive   Forum   Calendar   Search
MhonArc Date: [Date Prev] [Date Index] [Date Next]   Thread: [Date Prev] [Thread Index] [Date Next]   MhonArc
 

Re: [TEKNIK] Storke langsomt bash script [SOLVED]





Kristian Duus Østergaard skrev:
On 31/05/2010, at 11.28, Martin Dupont Ahrentsen wrote:

Hej i kloge folk bag keyboardet,

Jeg har et script som skal parse en en fil hvor den skal tjekke om nogle numre på nogle felter ligger indenfor nogle globale serier.

Filen t_pcl_ser indeholder ca 181.000 linier hvilket gør at det tager sindsygt lang tid at parse den.

formatet er som følgendde:

<snip>

Hej Martin,

Min erfaring er at til den type opgaver er awk een af de hurtigeste løsninger, så her er et lille awkscript som burde gøre det samme som dit bash script gem det som <navn>.awk
kald det med awk -f <navn>.awk <inputfil>   >    <outputfil>

Lad mig høre om det er blevet hurtigere end dit gamle.



BEGIN { FS="|";
		IFS="|";
		OFS="|";
		startno1=92080000001;
		endno1=92081327500;
		startno2=92085000001;
		endno2=92085135000;
		startno3=92085135000;
		endno3=92085973000;
	}

($2>=startno1 && $3<=endno1) ||
($2>=startno2 && $3<=endno2) ||
($2>=startno3 && $3<=endno3) {
			$6="Y";
			$7="Y";
	}
	{print $0}



Mvh.
Kristian Duus Østergaard

Hejsa alle,

Nu har jeg efterprøvet flere af de løsninger som er blevet forslået.
1.
       OLD_IFS=$IFS
       IFS="|"

       line=($i)
IFS=$OLD_IFS

       testcustomerid=${line[0]}
       teststartno=${line[1]}
       testendno=${line[2]}
       testactiveflag=${line[3]}
       testprocflag001=${line[4]}
       testprocflag002=${line[5]}
       testprocflag003=${line[6]}
       testprocflag004=${line[7]}
       testcountrya2=${line[8]}
       testzip=${line[9]}
	testprocflag005=${line[10]}

	Det gjorde at tiden blev skåret ned fra ~ 2 timer til 35 minutter

2.
       TCL løsningen fik jeg desværre ikke til at virke efter hensigten.

3.
	AWK løsningen som Kristian Duss Østergaard er klart vinderen. Process tiden her er nede på ca 1½ sekund!! Samtidig med at det var super let at integrere i det eksisterende script som også sørger for at distribuere output filen ud til ca 15 servere ved hjælp af awk -f <navn>.awk <inputfil>   >    <outputfil>.

BEGIN { FS="|";
		IFS="|";
		OFS="|";
		startno1=92080000001;
		endno1=92081327500;
		startno2=92085000001;
		endno2=92085135000;
		startno3=92085135000;
		endno3=92085973000;
	}

($2>=startno1 && $3<=endno1) ||
($2>=startno2 && $3<=endno2) ||
($2>=startno3 && $3<=endno3) {
			$6="Y";
			$7="Y";
	}
	{print $0}

I skal alle ha tak for hjælpen.


Med venlig hilsen

Martin Ahrentsen
Rødkælkevej 53, Hadbjerg
8370 Hadsten
25241017



 
Home   Subscribe   Mail Archive   Index   Calendar   Search

 
 
Questions about the web-pages to <www_admin>. Last modified 2010-07-01, 02:01 CEST [an error occurred while processing this directive]
This page is maintained by [an error occurred while processing this directive]MHonArc [an error occurred while processing this directive] # [an error occurred while processing this directive] *