Amazon S3
Amazon's S3 is a persistent storage system that is available for EC2 instances. While EC2 instances (or AMIs) lose their data when they are shut down S3 is an external data source that multiple EC2 instances can readily access. Each object stored in S3 can be up to 5 gigabytes in size, is stored in a specific Amazon region, and can be secured so only authorized people and applications can access them. Data on S3 is stored in user defined 'Buckets'. For details on S3 go to:http://aws.amazon.com/s3/.
S3 with Wowza Media Server
S3 is a compelling solution with Wowza Media Server because a user can put a single copy of a video file on S3 and have many Wowza Servers access and stream that file. So no matter how many Wowza Media Servers you are running you can have a single repository for the content. The S3 bucket is also persistent so while individual AMIs lose their data when they are turned off the files in S3 will still be available next time you start a server.
S3 Pricing
Pricing of S3 is set by Amazon and outside the scope of this document. The data streamed from S3 via the EC2 instance to your customers is charged at the regular Wowza rates. Amazon's pricing information on S3 can be found here: http://aws.amazon.com/s3/#pricing and Wowza Server for EC2 pricing is here:http://www.wowzamedia.com/ec2.html
Using S3 with a Wowza AMI
Managing your S3 Content
We recommended that you manage your S3 content is using a Firefox plugin called "S3Fox Organizer". You can get more information and download S3Fox Organizer here: http://www.s3fox.net/ Amazon does not have a built in function that allows S3 buckets to be mounted as a hard drive but there is an open source project called 's3fs' which is a FUSE-based file system which does allow S3 buckets to be mounted as a local file system read/write. This allows some interaction between the AMI Host OS and your S3 bucket but there are some performance caveats to be aware of.
To mount an S3 bucket on your AMI:
1) Open a Telnet connection to the instance, then create a new directory: (See the Wowza Media Server for EC2 Users Guide for details on using Telnet)
mkdir /mnt/s3
2) Then mount an S3 bucket like this:
/usr/bin/s3fs your.S3.Bucket -o accessKeyId=yourAccessKey -o secretAccessKey=yourSecretAccessKey -o default_acl=public-read /mnt/s3
Having the bucket mounted is an easy way to save your config files, etc... We advise against recording streaming video (live-record) directly to an s3fs mount.
Video on Demand with S3 (vods3)
Because of the latency of streaming video files from an S3 bucket the Wowza AMI includes an application designed to cache that content on the EC2 AMI itself. Any content stored on S3 should be streamed via this application, vods3.
The vods3 application is included in the default startup package. To stream video from S3 it is as simple as dropping your content file into a publicly accessable bucket and setting your stream name in the following format.
Where “ami-123456” is the current Wowza Server AMI id. To stream content, use stream names in the form:
[media-type]:amazons3/[s3-bucket-name]/[path-to-content-in-s3]
For example to play the file “mycoolvideo.m4v” that is stored in S3 bucket “mybucket” at the path “videos/coolvideos” from the Amazon instance “ec2-75-101-208-8.compute-1.amazonaws.com”, the connection information is:
Server: rtmp://ec2-75-101-208-8.compute-1.amazonaws.com/vods3
Stream: mp4:amazons3/mybucket/videos/coolvideos/mycoolvideo.m4v
The EC2 User's Guide covers streaming from S3 in detail. Please consult it for more configuration information including how to stream from a private S3 bucket.
How does the caching system work?
* The files are cached on the local instance the first time a stream is accessed.
* The system caches block by block to minimize start up times on first view.
* Items will be removed from the cache based on our caching algorithm.
-If the cache is not full then an item will live in the cache as long as it is being viewed.
-If no one is viewing the content then the time to live timer starts.
-If the item is not viewed for MaxTimeToLive milliseconds then it is flushed from the cache.
-If the cache is full and a new item wants to enter the cache then items that have been in the cache but have not been viewed for more than MinTimeToLive will be flushed from the cache to make room. Items are flushed in the reverse order from the last time they were viewed.
Recording live streams to S3
Based on user experience we no longer recommend recording live streams directly to an S3FS mount. Instead, save your recording to the local disk of the EC2 instance then transfer it to the S3FS mount once the recording is complete. For now you either need to move it manually or create a script to move the file. The issues with recording live streams to S3 are discussed in the following article: 'S3FS Fuse-based file system'.
1) Open a Telnet connection to the instance, then create a new directory: (See the Wowza Media Server for EC2 Users Guide for details on using Telnet)
mkdir /mnt/s3
2) Then mount an S3 bucket like this:
/usr/bin/s3fs your.S3.Bucket -o accessKeyId=yourAccessKey -o secretAccessKey=yourSecretAccessKey -o default_acl=public-read /mnt/s3
Having the bucket mounted is an easy way to save your config files, etc... We advise against recording streaming video (live-record) directly to an s3fs mount.
Video on Demand with S3 (vods3)
Because of the latency of streaming video files from an S3 bucket the Wowza AMI includes an application designed to cache that content on the EC2 AMI itself. Any content stored on S3 should be streamed via this application, vods3.
The vods3 application is included in the default startup package. To stream video from S3 it is as simple as dropping your content file into a publicly accessable bucket and setting your stream name in the following format.
Where “ami-123456” is the current Wowza Server AMI id. To stream content, use stream names in the form:
[media-type]:amazons3/[s3-bucket-name]/[path-to-content-in-s3]
For example to play the file “mycoolvideo.m4v” that is stored in S3 bucket “mybucket” at the path “videos/coolvideos” from the Amazon instance “ec2-75-101-208-8.compute-1.amazonaws.com”, the connection information is:
Server: rtmp://ec2-75-101-208-8.compute-1.amazonaws.com/vods3
Stream: mp4:amazons3/mybucket/videos/coolvideos/mycoolvideo.m4v
The EC2 User's Guide covers streaming from S3 in detail. Please consult it for more configuration information including how to stream from a private S3 bucket.
How does the caching system work?
* The files are cached on the local instance the first time a stream is accessed.
* The system caches block by block to minimize start up times on first view.
* Items will be removed from the cache based on our caching algorithm.
-If the cache is not full then an item will live in the cache as long as it is being viewed.
-If no one is viewing the content then the time to live timer starts.
-If the item is not viewed for MaxTimeToLive milliseconds then it is flushed from the cache.
-If the cache is full and a new item wants to enter the cache then items that have been in the cache but have not been viewed for more than MinTimeToLive will be flushed from the cache to make room. Items are flushed in the reverse order from the last time they were viewed.
Recording live streams to S3
Based on user experience we no longer recommend recording live streams directly to an S3FS mount. Instead, save your recording to the local disk of the EC2 instance then transfer it to the S3FS mount once the recording is complete. For now you either need to move it manually or create a script to move the file. The issues with recording live streams to S3 are discussed in the following article: 'S3FS Fuse-based file system'.
Comments
Post a Comment