encoding - Java―read, process and write UTF-8 file -

-

i trying write reads utf-8 encoded file can have encoding errors, process content , write result output file encoded in utf-8.

my program should modify content (kind of search , replace) , copy rest 1 one. in other words: if term search equals term replace, in- , output-file should equal well.

generally using code:

in = paths.get( <filename1> ); out = paths.get( <filename2> );  files.deleteifexists( out ); files.createfile( out );  charsetdecoder decoder = standardcharsets.utf_8.newdecoder(); decoder.onmalformedinput( codingerroraction.ignore ); decoder.onunmappablecharacter( codingerroraction.ignore );  bufferedreader reader = new bufferedreader(      new inputstreamreader(         new fileinputstream( this.in.tofile() ), decoder ) );  charsetencoder encoder = standardcharsets.utf_8.newencoder(); encoder.onmalformedinput( codingerroraction.ignore ); encoder.onunmappablecharacter( codingerroraction.ignore );  bufferedwriter writer = new bufferedwriter(      new outputstreamwriter(         new fileoutputstream( this.out.tofile() ), encoder) );  char[] charbuffer = new char[100]; int readcharcount; stringbuffer buffer = new stringbuffer();  while( ( readcharcount = reader.read( charbuffer ) ) > 0 ) {     buffer.append( charbuffer, 0, readcharcount );     //here goes more code process content     //buffer must written output on each iteration }  writer.write( buffer.tostring() ); reader.close(); writer.close();

but not working. compare files have little junit test fails:

byte[] bytesf1 = files.readallbytes( paths.get( <filename1> ) ); byte[] bytesf2 = files.readallbytes( paths.get( <filename2> ) ); asserttrue( bytesf1.equals( bytesf2 ) );

what doing wrong, or can working?

thanks in ahead, philipp

edit

unless manage make test work after ensuring input file encoded in utf-8, basic error, real point of interest , question is:

does approach above guarantee defects in utf-8 file copied 1 one, or process of loading chars stringbuffer change this?

java arrays don't implement value-based equals. fail:

asserttrue( bytesf1.equals( bytesf2 ) );

consider:

assertarrayequals(bytesf1, bytesf2);

or

asserttrue(arrays.equals(bytesf1, bytesf2));

Comments

Post a Comment

Popular posts from this blog

c# - How Configure Devart dotConnect for SQLite Code First? -

-
can configure following sample code ? when run following code, error : no entity framework provider found ado.net provider invariant name 'devart.data.sqlite' setting in machine.config <system.data> <dbproviderfactories> <add name="dotconnect sqlite" invariant="devart.data.sqlite" description="devart dotconnect sqlite" type="devart.data.sqlite.sqliteproviderfactory, devart.data.sqlite, version=4.6.287.0, culture=neutral, publickeytoken=09af7300eec23701" /> </dbproviderfactories> </system.data> after add below block code in app.config <?xml version="1.0"?> <configuration>   <configsections> <!-- more information on entity framework configuration, visit http://go.microsoft.com/fwlink/?linkid=237468 --> <section name="entityframework" type="system.data.entity.internal.configfile.entityframeworksection, entit...
Read more

java - Copying object fields -

-
what best way of copying classes when need have 2 independent variables? have simple class: public class myclass { boolean = false; string b= "not empty"; } do need make method : assign(myclass data ) { a= data.a; b= data.b; } is there automatic method copy (duplicate) objects in java? do need make method : pretty close. instead of making method, should make constructor. such constructors called copy constructor , , create them this: myclass(myclass data) { = data.a; b = data.b; } and create copy of instance, use constructor this: myclass obj1 = new myclass(); myclass obj2 = new myclass(obj1); copy constructor can tedious : using copy constructor create deep-copy can tedious, when class has mutable fields. in case, assignment create copy of reference, , not object itself. have create copy of fields (if want deep copy). can go recursive. a better way create deep copy serialize , deserialize object. why not use clone()? ...
Read more

c++ - Clear the memory after returning a vector in a function -

-
i have function processes , stores lot of data in , returns results vector of class. amount of data stored in function tremendous , want clear storage memory of function after finished job. necessary (does function automatically clear memory) or should clear memory function? update: vector<customers> process(char* const *filename, vector<int> id) { vector<customers> list_of_customers; (perform actions) return list_of_customers; } variables defined locally within function automatically released @ end of function scope. destructors objects called, should free memory allocated objects (if objects coded correctly). example of such object std::vector . anything you've allocated new must have corresponding delete release storage. try avoid doing own allocation , use raii instead, i.e. containers or smart pointers.
Read more