Workflow
Working with remote compute resources introduces an additional layer of complexity that must be integrated into a fluid workflow. There are several options which can help with this, each with their own benefits and
drawbacks:
1. use an editor hosted on the remote machine (VIM/EMACS):
(a) Benefits: very lightweight and it’s always there, at least on the OzStar/CIT clusters and on most Unix distributions. Can be easily customised, there are many helpful YouTube tutorials and provides a keyboard-only workflow that can provide a very fast way to interact with a codebase.
(b) Drawbacks: there are not many drawbacks resulting from the actual use itself but current IDE workflows provide additional functionality such as syncing local+remote code, version control and additional debug and code exploration tools.
2. use a contemporary IDE: PyCharm:
(a) Benefits: A contemporary IDE provides several key functional features that make it a great option for working with remote codebases. Some of these features are;
- Continous code synchronizations
- Version Control Management
- Remote interpreter/s
- Code navigation
- Integrated Debug and Prole features
(b) Drawbacks: Unlike a built-in lightweight editor, this has to be downloaded and properly setup before getting the above benefits. Some of the features are only available in the Professional version of PyCharm and may not be available in some other IDEs such as VsCode. Luckily, any .edu email holder qualifies for a free Professional version of PyCharm. Additionally, I’d recommend a minimum of 8gb of RAM for the machine hosting the distribution.
3. use a local text editor paired with an RSYNC or SCP custom pipeline:
(a) Benefits: you can continue to use your preferred editor such as Sublime or Textmate. These tend to be more lightweight and can easily run on machines with 4gb of RAM or even less. They often provide some code navigation aids and can integrate with version control subsystems.
(b) Drawbacks: they do not allow the use of remote interpreters and do not have advanced debug and/or profiler options. Additionally, the code syncing can be an issue if not properly monitored. In general, the additional general supervision required is generally offset by the initial time investment in just setting up an IDE rather than choosing this option.
Whichever option describes your workflow, there is no absolute right or wrong way but this document will focus on how to set-up a professional version of PyCharm to work with OzStar cluster resources