Tech Blog

Publish Flash Player 10 content

To test this, I first needed a way to make my Flex Builder compile Flash Player 10 content. I found a rather old post by Andrei Ionescu about building Flash Player 10 applications that told me exactly what I was looking for. I downloaded the latest Flex SDK (3.3) from Adobe’s Flex Download page, followed the tutorial and I was ready to go.

Load image from local file system

Reading a local file was no biggie, just copied and pasted the code from Mike’s sample. What I needed now was a way to convert the loaded data into an image. As usual Tim helped me out by sending me a snippet:

var data:ByteArray = fileRef['data'];
var loader:Loader = new Loader();
loader.loadBytes(data);
addChild(loader);

Modify image

Now that I could load the image, I wanted to modify it before having it rendered. I extended the snippet to use BitmapData and a Matrix to resize the image to a thumb image (240×180px).

var loader:Loader = new Loader();
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onDataLoadComplete);
loader.loadBytes(loadFileRef.data);
 
private function onDataLoadComplete(e:Event):void {
  var bitmapData:BitmapData = Bitmap(e.target.content).bitmapData;
  var matrix:Matrix = new Matrix();
  matrix.scale(THUMB_WIDTH/bitmapData.width, THUMB_HEIGHT/bitmapData.height);
 
  imageView.graphics.clear();
  imageView.graphics.beginBitmapFill(bitmapData, matrix, false);
  imageView.graphics.drawRect(0, 0, THUMB_WIDTH, THUMB_HEIGHT);
  imageView.graphics.endFill();
}

Save image to local file system

Now that it’s possible to load and modify an image, saving it is the last step. To save an image, it has to be encoded to a ByteArray. I used the open source as3corelib to help me out. Saving the ByteArray to a file is rather straight forward.

Update 1: You don’t need to use the as3corelib anymore, Flex 3.3 has it own JPEGEncoder.

var encoder:JPEGEncoder = new JPEGEncoder();
var rawBytes:ByteArray = encoder.encode(bitmapData);
 
var saveFileRef:FileReference = new FileReference();
saveFileRef.save(rawBytes);

Update 2: Thibault Imbert figured out a way to speed up JPEG encoding using the new FP10 Vector class. Good stuff!

Update 3: You can now also use Alchemy for the encoding. It’s much, much faster then the other options! See this post from Jens Krause: Speed up JPEG encoding using Alchemy

This sample shows that it’s not only possible to load, modify and save images directly from the local file system, but that it’s actually very simple to do so. Add some PixelBender image processing power to the game and you almost have a Photoshop killer running completly client side!

Code sample

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" backgroundGradientColors="[0xFFFFFF,0xFFFFFF]" backgroundColor="0xFFFFFF">
 
	<mx:Script>
		<![CDATA[
			import mx.graphics.codec.JPEGEncoder;
			import mx.binding.utils.BindingUtils;
			import mx.core.UIComponent;
 
			import flash.net.FileReference;
			import flash.net.FileFilter;
			import flash.events.IOErrorEvent;
			import flash.events.Event;
			import flash.utils.ByteArray;
 
			private var loadFileRef:FileReference;
 
			private static const FILE_TYPES:Array = [new FileFilter("Image Files", "*.jpg;*.jpeg;*.gif;*.png;*.JPG;*.JPEG;*.GIF;*.PNG")];
			private static const THUMB_WIDTH:uint = 240;
			private static const THUMB_HEIGHT:uint = 180;
 
			private function loadFile():void {
				loadFileRef = new FileReference();
				loadFileRef.addEventListener(Event.SELECT, onFileSelect);
				loadFileRef.browse();
			}
 
			private function saveFile():void {		
				var bitmapData:BitmapData = new BitmapData(THUMB_WIDTH, THUMB_HEIGHT);
				bitmapData.draw(imageView);
 
				var encoder:JPEGEncoder = new JPEGEncoder();
				var rawBytes:ByteArray = encoder.encode(bitmapData);
 
 				var saveFileRef:FileReference = new FileReference();
				saveFileRef.save(rawBytes);
			}
 
			private function onFileSelect(e:Event):void {
				loadFileRef.addEventListener(Event.COMPLETE, onFileLoadComplete);
				loadFileRef.load();
			}
 
			private function onFileLoadComplete(e:Event):void {
			   	var loader:Loader = new Loader();
			   	loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onDataLoadComplete);
			   	loader.loadBytes(loadFileRef.data);
 
				loadFileRef = null;
			}
 
			private function onDataLoadComplete(e:Event):void {
				var bitmapData:BitmapData = Bitmap(e.target.content).bitmapData;
 
                var matrix:Matrix = new Matrix();
                matrix.scale(THUMB_WIDTH/bitmapData.width, THUMB_HEIGHT/bitmapData.height);
 
				imageView.graphics.clear();
				imageView.graphics.beginBitmapFill(bitmapData, matrix, false); 
				imageView.graphics.drawRect(0, 0, THUMB_WIDTH, THUMB_HEIGHT);
				imageView.graphics.endFill(); 
 
				saveButton.enabled = true;
			}
 
		]]>
	</mx:Script>
	<mx:Panel title="Create thumb image" paddingBottom="5" paddingLeft="5" paddingRight="5" paddingTop="5">
		<mx:VBox>
			<mx:Canvas id="imageView" width="240" height="180" borderThickness="1" borderColor="#CCCCCC" borderStyle="solid"/>
			<mx:HBox paddingTop="5" borderThickness="1">
				<mx:Button label="Load image" click="loadFile()"/>
				<mx:Button label="Save image" click="saveFile()" id="saveButton" enabled="false"/>
			</mx:HBox>
		</mx:VBox>
	</mx:Panel>
</mx:Application>

I just updated Papervision3D to allow the CameraObject3D#unproject method to work when CameraObject3D#useProjectionMatrix = true.

Papervision3D has two methods of ‘projecting’ vectors onto the screen:

1. useProjectionMatrix = false, this is the default ‘fast’ method
2. useProjectionMatrix = true, this is the method where projection is done by a matrix

Note that Papervision3D switches to method #2 in ‘ortho mode’.

So what is the difference exactly between these two kinds of projection? First we need to define what projection is. Projection is what adds ‘perspective’ to a scene and makes sure your scene fits to the viewport. Its the last step in the so-called ‘render pipeline’.

We can derive the ‘fore shortening’ effect of perspective in several ways. Method #1 is simple, effective and fast. It uses the camera’s focus and zoom values. Method #2 uses a more classic way of projection : it uses a dedicated matrix. Tech buffs: very much like OpenGL’s gluPerspective.

Andy Zupko has a great post on unproject using method #1.

We will discuss unproject using method #2, which works slightly different.

The difference is that when #useProjectionMatrix is set to true, then CameraObject3D#unproject returns a point in world-space, rather then a ray. Let me explain with some code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// we want to use a projection matrix
camera.useProjectionMatrix = true;
 
// '0' indicates we want a point on the near plane
var pointOnNearPlane : Number3D = camera.unproject( screenX, screenY, 0 );
 
// '1' indicates we want a point on the far plane
var pointOnFarPlane : Number3D = camera.unproject( screenX, screenY, 1 );
 
// Construct the ray's direction
var dir : Number3D = Number3D.sub( pointOnFarPlane, pointOnNearPlane );
 
// Normalize
dir.normalize();
 
// So, now you have a ray with its origin at 'pointOnNearPlane',
// and direction 'dir'

Optimization: could save some cycles by doing (*not* in ortho mode!, see below):

1
2
3
4
5
6
7

var camPosition : Number3D = new Number3D( camera.x, camera.y, camera.z );
// Construct the ray's direction
var dir : Number3D = Number3D.sub( pointOnFarPlane, camPosition );
// Normalize
dir.normalize();
// So, now you have a ray with its origin at the camera's position
// and direction 'dir'

Why can’t we use above optimization in ortho mode?
In perspective mode all ‘rays’ originate from the camera’s position: the ‘view cone’ has shape of a pyramid. Hence we can assume ‘pointOnNearPlane’ to coincide with the camera’s position.
In ortho mode however the ‘view cone’ has the shape of a cube, all rays are parallel. Hence we need to unproject *two* points to construct our ray.

Now simply follow Andy Zupko ’s post to drag objects around.
Or perform ‘picking’, or…

Update:
Created a little demo to visualize what I’m ranting about, check it out here

We have been working hard to implement an XML-based REST-style API for Floorplanner and some of our partners are using it already to access their users and plans. We now have started to use this API ourselves.

The Floorplanner Flash application communicates with our servers to load and save projects and designs. The backend for this functionality used to be written in PHP. Eliminating this PHP application simplifies our server setup, eases development and reduces our maintenance burden. To rewrite this functionality in Rails, we decided to eat our own dogfood and use the REST API to load and save designs. Why reinvent the wheel?

However, while implementing the changes it the Flash application, we found that it did not work out of the box, because of some limitations in ActionScript. Note that we still use ActionScript 2, so some of these issues may not be relevant for ActionScript 3. In this post, we will detail what these issues are and how we overcame them.

Using a separate format

Our REST API uses the XML format supplied by Rails. Because we had to make some changes to make it work from ActionScript, we decided to add a distinct format that we could implement differently without altering the behavior of the default XML API. Adding a new format, called “flash” can be done in Rails by adding a MIME alias to your environment:

Mime::Type.register_alias "application/xml", :flash

Now we can send different responses based on this format:

respond_to do |format|
  format.xml   { ... } # default API behavior
  format.flash { ... } # do something different
end

HTTP status codes

REST APIs use HTTP status codes to return whether a call succeeded, and if not, why. Flash however uses the browser to perform HTTP requests. The browser only returns something to Flash if the request was successful. So, if an error code is used if a request failed together with some error messages, these error messages will not be delivered to Flash and cannot be displayed to the user. We resolved this by always sending the HTTP OK status.

respond_to do |format|
  format.xml   { render :xml => @project.errors, :status => 422 }
  format.flash { render :xml => XML.failure(@project.errors, 422) }
end

Our XML.failure method will return something like:

<failure status="422">
  <error on="name">A project should have a name!</error>
</failure>

Note that other HTTP success statuses than 200 work in Safari and Firefox, but not in Internet Explorer. So, never return a 201 (:created), because Internet Explorer will not send the result to Flash!

PUT and DELETE requests

REST-style APIs use HTTP PUT requests to alter objects and DELETE requests to destroy objects. Most browsers do not support these request type. It is not supported by ActionScript either, because Flash uses the browser to send the request.

To overcome this problem, these types of requests can be simulated in Rails by sending a _method parameter along with a POST request. Unfortunately, this does not work when calling the REST API. The POST request body cannot be used to send additional variables, because it is used for the XML payload.

We solved this issue by creating additional routes for POST requests to the update and destroy actions of our resource controllers. Our routes.rb file now looks like this:

  map.resources :projects, :member => { :update => :post, :destroy => :post }

These routes route to exactly the same methods as the default REST actions (ProjectsController#update and ProjectsController#destroy), so no additional code is needed. The following calls are now equivalent:

PUT    /projects/123.xml
POST   /projects/123/update.xml
POST   /projects/123/update.flash

DELETE /projects/123.xml
POST   /projects/123/destroy.xml
POST   /projects/123/destroy.flash

The result

It requires some stretching of the pure REST principles, but doing so is worth it: we can now reuse the code we use for our API to handle Flash application calls and we can eliminate the PHP backend.

Ted Patrick (Senior Manager Developer Communities at Adobe Systems) has posted an interesting article about the first milestone of the Eclipse E4 project.

It seems that the SWT project has added compilation support for SWF from JAVA. Write your app in JAVA and publish as SWF to Flash Player. The cool part is that you get full JAVA development in Eclipse with all debugging and tooling but you get a SWF file on publish. 

We’re using Flex Builder for our Flex/AS3 stuff, but this could become a very interesting option since the JAVA development in Eclipse is a lot more sophisticated than Flex Builder at the moment.

On February 22-24th the Flash In The Can event is back in town! Design, Technology. Cool shit. I will be just a visitor, but Tim is going to help Ralph with his workshop Papervision3D from the Core. If you want to meet up, give us a ping.

Last week we released a new Floorplanner account, the Enterprise account. With it, companies can have an online Floorplanner solution completely branded to their wishes. It includes the Roomplanner module and a custom library of their own furniture elements.  Steelcase and Maxon are the first to have an active Enterprise account.

The interesting part of this release, from a tech point of view, is the new drag-and-drop functionality. The Roomplanner is running in the Flash Player. However, the library of furniture elements is in HTML. We chose to build the furniture library in the HTML sidebar for a couple of reasons. It’s not in the Roomplanner, so there is more space left to design. It gives us more freedom in the page layout. All items on the page are indexed and things like sorting and tags we’re easier to build in Ruby on Rails than in Flash.

Now we only needed a way to get the furniture from the sidebar to the Roomplanner. That’s where the drag’n'drop comes in, drag from HTML and drop in Flash. To drag an image of a furniture element in HTML we used the fantastic JavaScript script.aculo.us framework. The next step was to swap the image to an element in the Roomplanner and to update the position of the element while moving the cursor on the Flash content. For this we used ExternalInterface to communicate between JavaScript and ActionScript. That’s all there is to it. The theory is actually quite simple, but in practice it was very (I repeat, VERY) difficult to get it working in all the main browsers. All those nitty gritty browser details…

At this moment everything seems to work just fine, so please take a look at Steelcase and/or Maxon and let me know what you think!

Today I learned a couple of new things about the use (and trouble) of BitmapData in combination with loading content from other domains. I thought we solved it a while ago, but it in some situations it didn’t work… I definitely hope this is my last post about the subject!

When you load content from another domain and you want to use BitmapData, then you always have to load a policy file (crossdomain.xml) from that other domain. What I discovered today is that you have to do this for ALL the content you’re loading, otherwise the BitmapData just doesn’t work. Even if it’s content you don’t want to use for the BitmapData, you still need to load the policy file. It took me quite a while of debugging to figure this out.

This is our current setup. Every server has its own domain.

The other thing I learned today is a small trick to remove the Warning in your logs. Jeffry Houser explains it in his post Fixing your CrossDomain.xml File.

UPDATE:
To fix the issue I discussed some days ago(see below) use:
System.security.allowDomain(”http://ip-adress”);
ip-adress is the IP you are developing on! This will fix the security issues!

While trying to let Actionscript talk to Javascript using the External Interface API of Actionscript I bounced upon a problem when Actionscript tried to invoke this Javascript function. In Firefox 2.0.0.2 an alert message shouted to me: “Error calling method on NPObject! [plugin exception: Error in Actionscript. Use a try/catch block to find error.].”.

My precise environment was, the html running on the localhost (Rails application) and the swf from an other domain and this seemed to be the problem. When I deployed the Rails application to this server, there was no problem at all invoking these functions. It looks like a bug in the flash player, cause I tried all the security settings possible in flash, the famous crossdomain.xml, adding all locations to the flash settings but no result. Somehow the SWF from that other domain had some troubles talking to the local javascript.

My way to solve this issue was to load the SWF also from the localhost. Everything works perfectly and I can continue testing.

After some diggin’ I finally figured out why Internet Explorer didn’t display my SWF movie. The width and the height of my movie are related to the width and height of the Stage. It seems that in IE the Stage isn’t available in the first frame. Therefore the movie has to wait a frame before it can acces the Stage properties.

public function Application() {
  addEventListener( Event.ENTER_FRAME, onEnterFrame );
}
private function onEnterFrame( pEvent:Event ) : void {
  removeEventListener( Event.ENTER_FRAME, onEnterFrame );
  setWidth( stage.stageWidth );
  setHeight( stage.stageHeight );
}