There are some things about Glue I absolutely love - it is highly scalable, cost effective and very easy to incorporate with other AWS services like Cloudwatch or Step Functions.
Performance issues exist in AWS Glue when dealing with UDFs as the data needs to be moved from the JVM to the Python process to be transformed and then moved back to the JVM.
I have been wanting to move over to Scala for development of Glue Jobs but always found it very difficult to test locally before deployment.
I have built my jobs using Maven rather than SBT. Here is the basic pom.xml file you will need to get started.
[
add
]
[
|
|
...
]