scala, transform a callback pattern to a functional style internal iterator -

-

suppose api given , cannot change it:

object providerapi {    trait receiver[t] {     def receive(entry: t)     def close()   }    def run(r: receiver[int]) {     new thread() {       override def run() {         (0 9).foreach { =>           r.receive(i)           thread.sleep(100)         }         r.close()       }     }.start()   } }

in example, providerapi.run takes receiver, calls receive(i) 10 times , closes. typically, providerapi.run call receive(i) based on collection infinite.

this api intended used in imperative style, external iterator. if our application needs filter, map , print input, need implement receiver mixes these operations:

object main extends app {   class myreceiver extends providerapi.receiver[int] {     def receive(entry: int) {       if (entry % 2 == 0) {         println("entry#" + entry)       }     }     def close() {}   }    providerapi.run(new myreceiver()) }

now, question how use providerapi in functional style, internal iterator (without changing implementation of providerapi, given us). note providerapi call receive(i) infinite times, not option collect in list (also, should handle each result 1 one, instead of collecting input first, , processing afterwards).

i asking how implement such receivertoiterator, can use providerapi in functional style:

object main extends app {   val iterator = new receivertoiterator[int]  // how implement this?   providerapi.run(iterator)   iterator     .view     .filter(_ % 2 == 0)     .map("entry#" + _)     .foreach(println) }

update

here 4 solutions:

updated: blockingqueue of 1 entry

what you've implemented here java's blockingqueue, queue size of 1.

main characteristic: uber-blocking. slow consumer kill producer's performance.

update: @gzm0 mentioned blockingqueue doesn't cover eof. you'll have use blockingqueue[option[t]] that.

update: here's code fragment. can made fit receiver.
of inspired iterator.buffered. note peek misleading name, may block -- , hasnext.

// fairness enabled -- want preserve order... // alternatively, disable fairness , increase buffer 'big enough' private val queue = new java.util.concurrent.arrayblockingqueue[option[t]](1, true)  // following block provides potentially blocking peek operation // should `queue.take` when previous peeked head has been invalidated // specifically, `queue.take` , block when queue empty private var head: option[t] = _ private var headdefined: boolean = false private def invalidatehead() { headdefined = false } private def peek: option[t] = {   if (!headdefined) {     head = queue.take()     headdefined = true   }   head }  def iterator = new iterator[t] {    // potentially blocking; false upon taking `none`   def hasnext = peek.isdefined    // peeks , invalidates head; throws nosuchelementexception appropriate   def next: t = {     val opt = peek; invalidatehead()     if (opt.isempty) throw new nosuchelementexception     else opt.get   } }

alternative: iteratees

iterator-based solutions involve more blocking. conceptually, use continuations on thread doing iteration avoid blocking thread, continuations mess scala's for-comprehensions, no joy down road.

alternatively, consider iteratee-based solution. iteratees different iterators in consumer isn't responsible advancing iteration -- producer is. iteratees, consumer folds on entries pushed producer on time. folding each next entry becomes available can take place in thread pool, since thread relinquished after each fold completes.

you won't nice for-syntax iteration, , learning curve little challenging, if feel confident using foldleft you'll end non-blocking solution reasonable on eye.

to read more iteratees, suggest taking peek @ playframework 2.x's iteratee reference. documentation describes stand-alone iteratee library, 100% usable outside context of play. scalaz 7 has comprehensive iteratee library.


Comments

Post a Comment

Popular posts from this blog

c# - How Configure Devart dotConnect for SQLite Code First? -

-
can configure following sample code ? when run following code, error : no entity framework provider found ado.net provider invariant name 'devart.data.sqlite' setting in machine.config <system.data> <dbproviderfactories> <add name="dotconnect sqlite" invariant="devart.data.sqlite" description="devart dotconnect sqlite" type="devart.data.sqlite.sqliteproviderfactory, devart.data.sqlite, version=4.6.287.0, culture=neutral, publickeytoken=09af7300eec23701" /> </dbproviderfactories> </system.data> after add below block code in app.config <?xml version="1.0"?> <configuration>   <configsections> <!-- more information on entity framework configuration, visit http://go.microsoft.com/fwlink/?linkid=237468 --> <section name="entityframework" type="system.data.entity.internal.configfile.entityframeworksection, entit...
Read more

java - Copying object fields -

-
what best way of copying classes when need have 2 independent variables? have simple class: public class myclass { boolean = false; string b= "not empty"; } do need make method : assign(myclass data ) { a= data.a; b= data.b; } is there automatic method copy (duplicate) objects in java? do need make method : pretty close. instead of making method, should make constructor. such constructors called copy constructor , , create them this: myclass(myclass data) { = data.a; b = data.b; } and create copy of instance, use constructor this: myclass obj1 = new myclass(); myclass obj2 = new myclass(obj1); copy constructor can tedious : using copy constructor create deep-copy can tedious, when class has mutable fields. in case, assignment create copy of reference, , not object itself. have create copy of fields (if want deep copy). can go recursive. a better way create deep copy serialize , deserialize object. why not use clone()? ...
Read more

gblinear xgboost in R -

-
Image
let's data has both numeric & catagoricial feature, , i've created xgboost model using gblinear. i've analyzed xgboost model xgb.importance, how can express categorical variable weights? while xgboost considered black box model, can understand feature importance (for both categorical , numeric) averaging gain of each feature split , trees. this represented in graph below. # feature real names names <- dimnames(trainmatrix)[[2]] # compute feature importance matrix importance_matrix <- xgb.importance(names, model = bst) # nice graph xgb.plot.importance(importance_matrix[1:10,]) in feature importance above, can see first 10 important features. this function gives color each bar. k-means clustering applied group each feature importance. alternately, represented in tree diagram (see link above).
Read more