Whether you like it or not, if you’re in the video production business, then you’re also in the business of data management. Video professionals must manage and protect a larger quantity of data than almost any other industry. That’s because video files are some of the largest file types around. When you’re shooting 8K RAW video at 60 frames per second, you’ll chew through terabytes of data in minutes.
And just like money, more data equals more problems.
Mo Data Mo Problems
When we first began operating years ago, we passed around drives in order to collaborate on video projects. While passing around drives is a simple solution for information storage and business workflow, it comes with inherent risks. For example, losing data is easy when you’re passing around drives. Each person has different habits when it comes to handling data. One person might decide to clear space on a drive and inadvertently delete important video footage. Or someone might overwrite a project file that, in turn, gets corrupted. Or, as a worst case, someone might even lose a hard drive altogether. All of these are common problems, but they don’t stop there.
As hard drives fill up, part of a project might have to be moved onto a separate drive. That means mounting multiple drives at once in order to connect all the media to the project. This makes the workflow cluttered and more complex than necessary.
Additionally, managing multiple drives, walking around to different offices to hand them off, and manually copying files from one drive to another all becomes part of a time consuming and high-risk workflow. At some point in your video production business life cycle, passing around drives will no longer suffice. Therefore, finding a better solution will become a necessity for survival.
There’s No Faster Way To Lose A Client Than By Losing Their Data
I have a saying that use often: “you’re only as good as the footage you backup.” No matter how much work you put into a shoot, if you fail to ensure that those files land in a drive then that project is a complete loss. And your time and effort aren’t the only things you’ll lose…you’ll likely lose the client as well. Because there’s no faster way to lose a client than by losing their data.
The harsh reality is that losing files is not a matter of “if,” but “when.” With thousands of files to manage across multiple drives, media cards, and over many years, losing files is inevitable for any video production company. Entertainment Creative Group is no exception to the rule. We have lost files. However, we adapted our workflow to mitigate this risk as much as possible. The solution: our SAN.
What is a SAN?
SAN is an acronym for storage area network. Simply put, it means that all storage is connected to a single computer on a local network where other computers on the network can access the files. There are many ways to build a SAN, and the prices fluctuate greatly. We needed to build a SAN with both a large capacity as well as a fast read and write speed, because both of those factors are extremely important in video production. Without getting into too much technical detail yet, our SAN has a capacity of 330TB at speeds faster than a typical SSD (solid state drive). All of our 15 suites are connected to this shared storage so that we never have to pass drives around again.
The Tech Specs
Our SAN configuration consists of 64 6TB server grade hard drives in RAID 60 configuration. Each of the four JBODs has 16 drives in RAID 6. Then the four JBODS are striped together RAID 10 for a final presentation of RAID 60. What this means is that up to two drives in each JBOD could fail without losing data. If three or more drives failed in the same JBOD, then we’d have a problem. However, it’s easy to see the drive health, and the system will notify us the moment a drive fails. So if that happens we can simply replace the bad drive with a new one.
Each JBOD connects to the RAID controller via four lanes of 48 Gigabit fiber. The RAID controller and JBODs are all connected to our dual processor server computer, which then connects to our 24 port 10 Gigabit managed switch via Cat 6A cable. All of that 10 Gigabit Cat 6A cable goes out into each suite throughout our studio. Furthermore, each one of our computers has a 10 Gigabit Network Interface Card so that it can take advantage of the full speed that the SAN provides through each Cat 6A cable run.
We keep our SAN room at a cool 68F for optimal server performance. The room has it’s own AC unit, a backup unit in case that one fails, and it’s own thermostat to monitor the temperature. Keeping the room cool is very important because you don’t want your system to overheat.
LTO-7 Backup System
Our SAN is backed up by a separate LTO-7 tape library server. This tape library is on an automated backup schedule that ensures that our SAN and LTO-7 backup library are congruent, which safeguards our data. If our SAN fails and data is lost, we could easily recover it using our LTO-7 backup.
The other benefit of the LTO-7 format is that the tapes are not RAIDed. That means its health isn’t dependent on the health of other tapes. The software automatically backs up and catalogues the data on each tape. With all this cataloged, we can point to the exact data we want to restore and the drive robot will load that particular tape. And then, it quickly, easily restores the data set back to our SAN.
Furthermore, LTO-7 is a magnetic tape format that’s designed to last much longer than a typical hard disk drive. For comparison, LTO-7 tapes are rated for 30 years before needing to be replaced. Conversely, a server grade HDD is rated for 5 years. This long lifespan makes LTO tapes ideal for long term file storage and archival.
SAN vs NAS
A lot of people tell me about their NAS builds and ask me what the difference is between a NAS and a SAN. NAS is network attached storage whereas SAN is a storage area network. The major distinction is that a NAS is connected to the internet/cloud and accessible from anywhere, whereas a SAN is only available on a local intranet. There are pros and cons to this. We chose to go the SAN route because we don’t want our data connected to the internet. Having data connected to the internet exposes it to more risk from hackers and viruses.
Also, a SAN is typically much faster than a NAS, because you aren’t limited to the speed of your internet connection. For example, say you have a Gigabit speed internet connection for both upload and download (a rarity these days). If you were connecting to that NAS remotely, then your maximum throughput would be limited to 1Gbps or 125MB/s. Compare this speed to the real world speeds that we get on our SAN, which is approximately 700MB/s write and 600MB/s read.
That said, we know you have to work remotely at times.Our clever solution that lets us access our SAN from the cloud without exposing it to undue risk? Using Chrome Remote Desktop to control our computers remotely. Of course, the computer must be on and connected to the internet. But, as long as that’s the case, it allows us to puppet our computers and perform many tasks remotely.
How Our SAN Has Improved Our Workflow
Our investment in our SAN has impacted our company’s workflow more than any other technological investment in our history. It allows us to work more efficiently and securely, while also removing the burden of constantly backing up data manually. In short: no more stress about whether or not our data is safely stored.
Before we had a SAN, people avoided handling media management tasks. The mental burden and responsibility was tremendous, since if data was lost, you were the person to blame. Now, all of that is automated and streamlined, which allows us to focus on what we do best – making great video content!
Furthermore, our SAN lets us work better as a team. With all of the files in one place, all our work happens in real-time and at speeds faster than an SSD. We have folders that contain mass amounts of shared stock footage, music, graphics, fonts, LUTs, etc. Moreover, we can see all of the updates that other departments make to a project in real time. And we can even use ProjectStore to work within the same project together! Working in the same project with other editors is really beneficial for fast-moving projects like reality TV shows. It allows editors to work from the same bin structure, borrow assets from other sequences, and see what other editors are working on in their sequences/segments.
Lastly, our SAN and LTO-7 system has allowed us to provide professional data backup services for our clients. This provides ECG with another revenue stream while also giving clients peace of mind knowing that their data is safe with us.
We’ve gone through a lot in this rather technical blog, so please reach out in the comments if you have questions or advice. Thanks for reading!
2 Responses
Wow. Video media storage requires so much space. Did you build the SAN yourself? How much does a video storage solution cost?
I did a ton of research and spec’d out all of the tech but it was Electronic Brain in Florida who built it. They’re a great support partner and their pricing is very reasonable. That said, this sort of endeavor isn’t cheap. First, we had to run the 6A cable throughout the office and that coat about $4K. Then we built the 330TB SAN system which cost about $80K. And the LTO7 system cost about $60K.