Have you ever wondered what it takes to work in data and analytics? What background should you have? What skills or technical expertise? How do you know if you’ll like the work?
Whether you’re outside the field looking in or inside looking for a new challenge, you’ll benefit from our new Q&A series. Every other Friday, our own Informationists will share their thoughts, experiences, and advice on what they do and what they did to get there. Expect to see a wide range of answers from individuals in the same lines of work; our Informationists come from all walks of life, which only shows that there’s more than one way to get on the right career track.
Kicking off the new series is Dip Kharod, Big Data Architect. Dip earned his MS in Computer & Information Science from the New Jersey Institute of Technology. He has worked across the healthcare, life sciences, and financial services industries in the United States (including New York City, Kansas City, and Minneapolis), as well as in the United Kingdom (London).
What does a Big Data Architect do?
In a nut shell, a Big Data Architect designs and builds efficient yet cost-effective Big Data applications to help clients answer their business questions. Sounds like what Enterprise Architects or Solution Architects typically do? But Big Data Architects are required to look at traditional data processing problems from different lenses (e.g., How do I build a platform that provides just enough information in the hands of the business to make timely decisions while processing a massive amount of data that allows advanced analytics to answer never-before-asked questions in a secure environment?)
What educational and/or professional background does a Big Data Architect need?
What helps me every day is a number of years of experience in information management and application development built on top of a strong engineering and science background. Over the years, I’ve had opportunities to work in multiple industries with a variety of technologies, ranging from Tcl/Tk, .NET, J2EE, to Hadoop and NoSQL DB! At the end of the day, experience is the king!
What skills do you need to be a Big Data Architect?
You definitely need to be strong with information management and data processing on multiple platforms. You should know application design, and I prefer hands-on development experience. And you need to have an open mind in solving old and new problems.
What tools and technologies does a Big Data Architect use?
I use a lot of different tools for different purposes. For example, I use modeling tools, such as ERWin, Enterprise Architect, and Visio, as well code in core Java and scripting languages, such as Shell script, Python, etc. In the Hadoop ecosystem, I use Hive, Pig, MapReduce, and other tools, as well as NoSQL databases when necessary. You should also know about the different available search solutions, like Solr, ElasticSearch, etc. I even use RDBMS. (Yes, RDBMS! There are cases where business needs are best served with traditional RDBMS.)
What are the common traits of the best Big Data Architects?
You have to love data. Lots of data. Of good and bad quality. It also helps to be nimble, especially when it comes to newer technology. You should be judicious about tool selection and have the ability to embrace open-source technologies, with all their good and challenging aspects.
What are your favorite and least favorite parts of being a Big Data Architect?
I love addressing the challenges presented by the massive amount of data being generated every day and emerging advanced analytics. Implementing new solutions for clients is the best part of the job! As an Architect, building very efficient yet cost-effective platforms has always been a challenge and the expectations that come with Big Data make the job even more interesting!
What advice do you have for someone looking to become a Big Data Architect?
Big Data is a huge field, and it’s natural to get lost with so many new tools and technologies. So my suggestion would to grab the first opportunity that you get. Press on and more opportunities will follow. Remember, persistence is the key to success!
Have other questions or comments for Dip? Post them in the comments!