March, 23, 2020

Scala for Data Science

Scala for data science. These words may fill many data scientists with fear and uncertainty--others may even react with indignation. After all, doesn’t our Python toolkit give us everything we need for exploring data, training algorithms, visualizing results, and making discoveries? What is the benefit of throwing a new language into the mix, especially one with a decidedly less intuitive feel for many of us coming from academic research backgrounds? 

Continue Reading
March, 10, 2020

Why I Love Scala, Part I

While looking for an internship in late 2015, I stumbled across Scala, and was intrigued. Why would I want to use it over Java, especially since Java 8 added lambdas and streams? What made it better? I decided to try it out the following spring break, and I was instantly in love, so much so that I started contributing to the language the next year.

Continue Reading
February, 26, 2020

Scaling ETL with Scala

When I joined Protenus in 2015, the first version of our ETL “pipeline” was a set of HiveQL scripts executed manually one after another. The company, still a start-up focused on proving out the analytics and UX, had adopted Spark, Hive, and MongoDB as core technologies. With our Series A funding round completed, my first task was to take these scripts and build out an ETL application. The first attempt naturally adopted Spark and Hive as primary technologies and added state management. This version got us through our next few clients.

Continue Reading
February, 3, 2020

Scaling Infrastructure for Growth

I joined Protenus in 2017, just as we were maturing from the startup to growth phase of the business. A key improvement identified to drive future growth during this period was the ability to decrease the time it took to onboard new customers. This presented some interesting infrastructure challenges as, until this point, hosts and resources in our Amazon Web Services (AWS) account had been manually provisioned and configured to meet requirements set forth by various HIPAA rules and our stringent security policies.

Continue Reading
January, 20, 2020

Software Development Life Cycle at Protenus

Protenus’s software development life cycle is the process through which our platform is created. Its progression can be roughly described as the following phases: planning, creating, testing, deploying, and maintaining. At Protenus, these phases are flexible and often overlap. Let’s have a look at each phase from the software engineer’s point of view to see how they unfold.

Continue Reading