Asynchronous data/image loading

From Processing

Jump to: navigation, search
Versions: 1.0+
Contributors: toxi
Started: 2008-06-27


Here we describe a technique to load a number of files asynchronously in a separate thread without blocking the main application flow. Since the files are loaded as byte arrays the classes can be used for any type of media and basic handlers are provided to convert/retrieve the loaded data in a specific type (e.g. as text or image).

Caching

Depending on the usage scenario of your application this solution also supports the in-memory caching of files, so that if caching is enabled, the files are made available immediately without having to be reloaded from disk or online. For example this behavior was desired for the London College of Fashion Graduate Exhibition.

The cache library used is WhirlyCache, which is included in the demo downloads below.

General Approach

In order to handle the data loading without interrupting the main application, we need to run it in its own thread. A thread is a separate process, which will run independently until it's finished. Because this new thread will need to communicate with main application (e.g. there might be errors) we also need a mechanism to relay messages. In Java the best and most flexible way to do this is via an interface which consists of a number of possible callback functions (see below). In this case there're only 3 possible events:

  • itemLoaded - called each time a file has been downloaded successfully
  • itemFailed - called each time there was an error with downloading a file
  • queueComplete - called to inform the listener that the queue has been fully processed

In order to receive these events we need to create a little utility class which implements these methods. Please see the demo below for further reference.

Contents

Source Code

The solution consists of 3 parts:

1. DataLoadCallback

This interface needs to be implemented by the owner class of the load queue in order to receive status events. You can see how this is used in the demo below… 

/**
DataLoadCallback taken from http://wiki.processing.org/w/Asynchronous_data/image_loading
@author toxi
*/
public interface DataLoadCallback {
  public void itemLoaded(DataBuffer buffer);
  public void itemFailed(Exception e);
  public void queueComplete(DataBuffer[] buffers);
}


2. CachedDataLoadQueue

This class is handling the actual loading in its own thread. You can use this class to download any type of file (locally or URLs)… Before starting the queue you need to assign a listener class for callback events. Again, see the demo for an example.

The code has a dependency on WhirlyCache, but its use is optional. If you don't want to use caching, simply use the 2nd version of the constructor which doesn't take a cache reference. 

/**
CachedDataLoadQueue taken from http://wiki.processing.org/w/Asynchronous_data/image_loading
@author toxi
*/
 
class CachedDataLoadQueue extends Thread {
  private DataLoadCallback caller;
  private Cache cache;
  private PApplet app;
  private String[] files;
  private DataBuffer[] buffers;
 
  private boolean isAlive = true;
 
  public int sleepTime = 30;
 
  public CachedDataLoadQueue(PApplet app, String[] files, Cache c) {
    this.app=app;
    this.files=files;
    buffers=new DataBuffer[files.length];
    cache=c;
  }
 
  // use this constructor if you don't need/want caching
  public CachedDataLoadQueue(PApplet app, String[] files) {
    this(app,files,null);
  }
 
  public void setCallback(DataLoadCallback cb) {
    caller=cb;
    if (caller==null) isAlive=false;
  }
 
  // some web APIs require a minimum delay between requests from the same IP
  // you can set this duration here (in milliseconds)...
  public void setSleepTime(int delay) {
    sleepTime=delay;
  }
 
  // main thread loop, triggered by the start() method
  public void run() {
    int id=0;
    while(isAlive && id<files.length) {
      try {
        String itemKey=files[id];
        DataBuffer buffer=cache!=null ? (DataBuffer)cache.retrieve(itemKey) : null;
        if (buffer==null) {
          byte[] bytes=app.loadBytes(itemKey);
          if (bytes!=null) {
            buffer=new DataBuffer(bytes);
            buffers[id]=buffer;
            if (cache!=null) cache.store(itemKey,buffer);
            if (caller!=null) caller.itemLoaded(buffer);
            Thread.sleep(sleepTime);
          } 
          else {
            isAlive=false;
            Exception e=new IOException("file not found: "+itemKey);
            if (caller!=null) caller.itemFailed(e);
            else e.printStackTrace();
            return;
          }
        } 
        else {
          buffers[id]=buffer;
        }
        id++;
      }
      catch(InterruptedException e) {
        break;
      }
    }
    if (isAlive) {
      if (caller!=null) caller.queueComplete(buffers);
    }
  }
 
  public DataBuffer[] getResultBuffers() {
    return buffers;
  }
}

3. DataBuffer

This is a convenience wrapper for the raw data of downloaded files in order to support various media types in the CachedDataLoadQueue class… The queue will pass these DataBuffer objects to the listening class which can then make a decision about which media type to use for the data. 

/**
DataBuffer taken from http://wiki.processing.org/w/Asynchronous_data/image_loading
@author toxi
*/
 
import java.io.*;
import javax.imageio.*;
import java.awt.image.BufferedImage;
 
import processing.core.PImage;
 
public class DataBuffer {
 
  private byte[] bytes;
 
  public DataBuffer(byte[] buf) {
    bytes=buf;
  }
 
  public byte[] getRaw() {
     return bytes;
  }
 
  public String getAsText() {
    return getAsText("UTF-8");
  }
 
  public String getAsText(String encoding) {
    try {
      return new String(bytes,encoding);
    } 
    catch(Exception e) {
      e.printStackTrace();
    }
    return null;
  }
 
  // updated according to:
  // http://forum.processing.org/topic/converting-bufferedimage-to-pimage#25080000000340208
  public PImage getAsImage() {
    try {
      ByteArrayInputStream bis=new ByteArrayInputStream(bytes); 
      BufferedImage bimg = ImageIO.read(bis); 
      PImage img=new PImage(bimg.getWidth(),bimg.getHeight(),PConstants.ARGB);
      bimg.getRGB(0, 0, img.width, img.height, img.pixels, 0, img.width);
      img.updatePixels();
      return img;
    }
    catch(Exception e) {
      System.err.println("Can't create image from buffer");
      e.printStackTrace();
    }
    return null;
  }
}

Demo sketch

As demo we will asynchronously download a bunch of images from flickr and then display them as slideshow. Note that the main draw() loop is still running at full speed whilst the images are downloaded in the background… 

/**
Flickr slideshow demo taken from http://wiki.processing.org/w/Asynchronous_data/image_loading
@author toxi
*/
 
QueueListener listener;
 
int currID=0;
 
void setup() {
  size(500,375);
  // list of images to load for the slideshow
  String[] files=new String[]{
    "http://farm4.static.flickr.com/3109/2477299178_d477ab631e.jpg",
    "http://farm3.static.flickr.com/2177/2477309902_703837b9f1.jpg",
    "http://farm3.static.flickr.com/2380/2477310788_cb4376cb53.jpg"
  };
  // create & start image loader
  listener=new QueueListener(this, files);
}
 
// the main loop will execute at full speed until the images are loaded
void draw() {
  if (listener.isMediaLoaded) {
    // images are downloaded, now cycle through them... 
    frameRate(1);
    background(0);
    image(listener.images[currID],0,0);
    currID=(currID+1)%listener.images.length;
  } else {
    // anything in here is executed during image loading
    stroke(random(255));
    line(0,frameCount%height,width,frameCount%height);
  }
}
 
// this is our little listener class which kicks off
// and monitors the actual image loading
class QueueListener implements DataLoadCallback {
 
  CachedDataLoadQueue queue;
  boolean isMediaLoaded=false;
  PImage[] images;
 
  QueueListener(PApplet app, String[] files) {
    queue=new CachedDataLoadQueue(app,files,null);
    queue.setCallback(this);
    queue.start();
  }
 
  // this method is called after a file has been downloaded successfully
  void itemLoaded(DataBuffer buffer) {
    println("itemloaded: "+buffer);
  }
 
  // called each time there was an error with downloading a file
  void itemFailed(Exception e) {
    println("itemFailed: "+e);
  }
 
  // final callback to say the whole queue has been processed
  void queueComplete(DataBuffer[] buffers) {
    println("queueComplete: "+buffers.length+" loaded");
    images=new PImage[buffers.length];
    for(int i=0; i<buffers.length; i++) {
      images[i]=buffers[i].getAsImage();
    }
    isMediaLoaded=true;
  }
}

Downloads

Media:dataqueueloaderdemo.zip (I can't upload this file right now, there's a 2MB limit.)

Related Links


See also

Retrieved from "http://wiki.processing.org/w/Asynchronous_data/image_loading"
Personal tools