[an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] (none) [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] (none) [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive][an error occurred while processing this directive]
 
[an error occurred while processing this directive] [an error occurred while processing this directive]
Skåne Sjælland Linux User Group - http://www.sslug.dk Home   Subscribe   Mail Archive   Forum   Calendar   Search
MhonArc Date: [Date Prev] [Date Index] [Date Next]   Thread: [Date Prev] [Thread Index] [Date Next]   MhonArc
 

Re: [TEKNIK] Storke langsomt bash script



On 31-05-2010 11:28, Martin Dupont Ahrentsen wrote:
> Hej i kloge folk bag keyboardet,
>
> Jeg har et script som skal parse en en fil hvor den skal tjekke om
> nogle numre på nogle felter ligger indenfor nogle globale serier.
>
> Filen t_pcl_ser indeholder ca 181.000 linier hvilket gør at det tager
> sindsygt lang tid at parse den.
>
> formatet er som følgende:
>
> 2080099992|92091661701|92091661800|N|N|Y|Y|N|DK||Y|
> 2080099997|09999700000|09999799999|N|N|Y|N|N|DK|6000|Y|
> 2080099997|92091293701|92091293800|N|N|Y|N|N|DK|6000|Y|
> 2080099999|09999900000|09999999999|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92092380001|92092380100|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92093628201|92093629200|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92094989701|92094990200|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92094992601|92094992700|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92094992701|92094992800|N|N|Y|Y|N|DK|6000|Y|
> 2080099999|92095071401|92095071500|N|N|Y|Y|N|DK|6000|Y|
>
>

Nu er bash scripts ikke min stærke side, men det første der slog mig er
at du laver en cut for hvert flet, dvs scanner hver linje flere gange.

Nu er det ikke Java dette drejer sig om, men jeg har tidligere lavet en
konvertering af flade filer, komma separeret og fast kolonne bredde med
over 26 millioner linjer. Det tog 10 minutter på en laptop. Så Bash
burde kunne klare det bedre med kun 181000 linjer :)

Mvh
Asbjørn Grandt


 
Home   Subscribe   Mail Archive   Index   Calendar   Search

 
 
Questions about the web-pages to <www_admin>. Last modified 2010-07-01, 02:01 CEST [an error occurred while processing this directive]
This page is maintained by [an error occurred while processing this directive]MHonArc [an error occurred while processing this directive] # [an error occurred while processing this directive] *