Today, we are pleased to announce that the HDInsight tool in Azure Toolkit for Eclipse is generally available. Since our GA announcement of HDInsight Tool for IntelliJ, we are maintaining our strong momentum to serve the open source community and expanding our support for Eclipse. The HDInsight Tool for Eclipse is part of the Azure Toolkit for Eclipse and will be of particular interest to HDInsight Spark developers. It is an open source project as part of Azure Toolkit for Eclipse, whose source code is available under the MIT License from the project’s site at: http://bit.ly/29l1YnJ.

The HDInsight Tool for Eclipse is intended to improve usability, boost productivity of Spark developers and enable you to enjoy native Scala and Java development experiences from Eclipse. For big data developers building Spark applications on HDInsight, it can sometimes be hard to get started building your first application, and the Spark application development lifecycle in IDE can be long and tedious. With this tool, you can get started with HDInsight Spark in just a few minutes and experienced Spark developers can iterate their development cycles faster and easier.

What features are supported in HDInsight Tool for Eclipse?

The HDInsight Tool for Eclipse extends Eclipse to allow you to create and develop HDInsight Spark applications and easily submit Spark jobs to Microsoft Azure HDInsight Spark clusters using the Eclipse development environment. It integrates seamlessly with Azure, enabling you to easily navigate HDInsight Spark clusters and to view associated Azure storage accounts. To further boost productivity, the HDInsight Tool for Eclipse also offers the capability to view Spark job history and display detailed job logs. 

The HDInsight Tool for Eclipse offers the following key capabilities:

•    Create Spark projects with built-in templates, sample code and intelligence to autocreate artifacts and locate assemblies. 
•    Develop Spark applications with native authoring support (e.g. IntelliSense, auto format, error checking, etc.) from Eclipse (mainly Scala and Java).
•    Submit Spark applications to HDInsight clusters.
•    Allow users to stop a running Spark application before its completion.
•    Test and validate Spark applications locally.
•    View all the Spark clusters associated with a user’s Azure subscriptions.
•    Navigate the associated storage resources of a user’s HDInsight Spark cluster.
•    Cache the Spark logs to disk in case the Spark logs are too big.

How do I get started?

You need to first install Eclipse and download the prerequisite files including Java SDK, Scala IDE for Eclipse and Microsoft HDInsight Spark Cluster, then get the latest bits by going to the Eclipse repository and searching “Azure Toolkit for Java”. Eclipse will also prompt you for the latest updates if you have previously installed the Azure Toolkit for Java plugin.

For more information, check out the following links:

•    Video: Introducing HDInsight Tools for Eclipse for HDInsight Spark Development
•    Documentation: http://bit.ly/29xQ11T

Learn more about today’s announcements on the Azure Data Lake Blog.

Discover more Azure service updates.

If you have questions, feedback, comments, or bug reports, please use the comments below or send a note to [email protected].