In general, integration work is difficult because multiple moving parts make it easier to break. Now, apply that concept to a financial industry brimming with regulation, strict security protocols, little room for error, and evolving continuously, and you have one of the most complex disciplines in technology. I asked two of our integration masters, Eugene Berman, Senior Solutions Architect, and Adrian Hsieh, Director of Professional Services, to share their favorite tools, troubleshooting techniques, customer education skills, and other approaches that they have perfected over the course of decades working in integration. Here’s what they had to say:
What is the most common cause of problems during development and deployment?
Eugene:
There are two types of problems in the world of electronics; a good connection when you don’t want one and no connection when there should be. This may be an industry joke, but it’s shockingly true for software integration too; either you don’t get the data or a connection that you need or you get the data that you don’t need. So, it all comes down to what information you get access to and at what point in the development process you get it.
When something goes wrong in integration, the most common causes that we check first are whether the connected systems are still live and still in the same IP address or URL. Next, we ensure that the code has maintained proper credentials and look to see if they still expose the same data or if formatting has changed.
Adrian:
When a deployment fails due to missing information, it often goes beyond simply having the correct credentials against the end system. It can be missing or using improper parameters for a particular endpoint. Since the values are stored separately in secure stores in the cloud, spotting the error during configuration review is hard. People make mistakes, so the wrong value can pass through when they create their Helm charts and YAML files for deployment. In my experience, missing or mistaken credentials are most often the result of human error during deployment.
What are your favorite tools for development/troubleshooting, and why?
Eugene:
I grew up before the glory days of the computer; no Windows, no mouse, nothing. All I had was a simple command line or a batch job that I could submit and wait in line for it to execute. So, I naturally favor command line tools over anything else for their time-proven simplicity and reliability. For example, cURL is an incredibly valuable command line tool for checking HTTP or HTTPS-based data sources. A couple of other tools in the same category are standard UNIX tools like Ping and Traceroute, which solve connectivity problems by ensuring that an address is correct and reachable.
Wireshark is a more sophisticated tool I use to intercept and investigate TCP traffic and TCP packets to determine whether they are what we expected. Obviously, that won’t work with SSL, but you can see that your request was made correctly and observe the initial packets coming in. Their simplicity makes these tools very powerful but they require an in-depth knowledge of the protocols and technologies you’re using.
Lastly, no matter which programming language you work with, there are editors, IDEs, and text editors that can be great tools at your disposal. At PortX, we use IntelliJ, an incredibly powerful IDE initially designed for Java. We like it because it has plugins that allow you to work with much more than just Java. We used it to build the debugger tool for Camel. It’s also a relatively lightweight, high-performance tool that’s easy to learn and easy to use.
Adrian:
Eugene mentioned a great list of tools. Is it okay to say that my favorite tool is Slack? Honestly, we have an incredible team, and Slack is an unparalleled communication tool that allows me to solve challenges with people like Eugene.
If there was one tool or technology possible that doesn’t exist today, what would it be?
Eugene:
You know, I always wanted that communicator tool from Star Trek. Oh wait, we already have that. I guess I’ll have to settle for time travel.
Seriously, I think we have everything we need. The technology at our disposal is more powerful than what we use it for. The industry needs to apply the technology we have to create more efficient methods of working.
The Debugger that we recently contributed to Apache Camel was not a tool that existed, nor did we absolutely need it to build PortX. But it is a tool that will make us far more productive. The Camel platform is very powerful; it has everything that we need. But we applied our knowledge of the platform to build the debugger, making it easier and more time-efficient to use.
What makes integration more challenging than other software development?
Adrian:
Integration is more challenging because there are more moving parts involved, which makes the likelihood of one of those parts changing much higher. Software platforms are not static. There will be upgrades, patches, and configuration changes.
For integration to work properly, you need to consider the entire ecosystem to build in the proper resilience and robustness that allow for changes. And, it’s important that the integration includes sufficient monitoring and resilience so that when something changes in one of the connected applications, it informs the team to make adjustments readily, allowing the system to adapt quickly.
The knowledge and experience to pinpoint where the error occurred and forecast where it will occur is a skill that takes time to acquire. Many developers fresh out of school don’t have that skill. As it becomes more difficult and expensive to hire senior developers with this knowledge, a tool that does this for you is particularly valuable.
What makes integration for the financial services industry even more challenging?
Adrian:
The challenges are compounded in financial services because the industry requires higher accuracy and reliability. The consequences of a mistake are much higher in the financial industry. In the US, it’s governed by very strict regulations and security compliance is super important. Your whole integration flow must be hacker-proof because there is much at stake if it leaks sensitive information. The nature of the industry demands higher expectations of the quality of the work.
Eugene:
There is also the non-technological challenges of working in the financial integration space. A good system integrator is expected to understand the client’s business requirements and the nature of the data that’s coming in and going out. The customer doesn’t always accurately know what they need. They’ll tell you what they want.
The difference between mediocre system integration companies and the best consultants is the deep industry understanding to effectively educate the customer about how their business works and guide them through a solution that they actually need. Essentially, this requires you to become an expert in the industry, and financial services, in particular, is a complex industry to learn.
Lastly, the majority of banking core vendors that we work with lack proper documentation or omit it completely. If it is available, it’s often poorly written, and gaining access requires some teeth pulling and jumping through hoops.
Historically, banking core vendors haven’t had a reason to design their solutions with integration in mind. I think we are starting to see that change slowly. There are some cores that do include nice APIs. However, most have no APIs and there are a handful of solutions that offer something they market as APIs because banks and credit unions are catching on. The waters are murky here. FIs should evaluate any “middleware” solution offered by core vendors because most of the time, it’s proprietary software that further locks the financial institution into that solution.
What are the approaches and methodologies that help to overcome these challenges?
Adrian:
At PortX, this starts with our API-led approach for integration development. The API-led approach complements nicely with the agile methodology we also follow. We have an unwavering dedication to apply best practices in everything we do.
Moreover, since we specialize in serving small and medium-sized credit unions, banks, and fintechs, we have specifically applied the API-led and agile methodologies to develop solutions in the financial services world. The big integration vendors like MuleSoft and Dell Boomi don’t specialize in any given industry, so they can’t fine tune their work to a specific vertical.
We are constantly improving upon our methodology too. After each new project, we apply the lessons learned to improve our PortX platform and standardize the implementation process for our customers. Every time we do this, the process becomes faster with fewer errors.
Eugene:
Just to add to what Adrian is saying, we are “organically” building a library of cores and application integrations so that each new project that uses the same core can be delivered in a fraction of the time. For example, our partner, Narmi, had a new customer that was using a core that we have already worked with. We were able to build off of the solution that had already been created. We spent some time understanding if there were any gaps in the functionality for this specific client and we were able to quickly make improvements to that API.
And, our Narmi-facing API doesn’t ever need to be changed unless they alter the way they call PortX. This approach to reusability is especially powerful in helping our fintech partners scale.
It’s hugely beneficial to FIs also. This goes beyond having a solution already built and proven to work. They also realize ongoing advantages as we continuously improve upon the existing solution and redeploy.
Keep the conversation going
Our work with banks, credit unions, and fintechs has made us experts in the financial industry. However, our customers are the ultimate industry specialists, and we are constantly looking to improve our products and services to serve them better. Please use the comment section below to tell us if we missed anything in this interview that you would like to see on a future blog or webinar. If you have questions or would like to learn more, don’t hesitate to start a conversation with a member of our team today.
- Adrian Hsieh is the Director of Professional Services at PortX with more than 25 years of engineering experience. At PortX, Adrian’s primary focus is to ensure that we develop and deliver the right solutions to meet our customers’ requirements successfully. He also works closely with the PortX product team to continuously improve our offerings to suit our customers’ needs. Prior to joining PortX Adrian worked at many other technology companies such as Collibra, MuleSoft, Salesforce.com, and Blue Martini software. Adrian has a doctorate degree in Aerospace Engineering from the University of Michigan and a Master’s degree in E-Commerce from Carnegie Mellon University.
- Eugene Berman is a Senior Solutions Architect at PortX with 27 years of software engineering experience with integration frameworks. Before joining PortX, Eugene served nearly ten years as a Solution Architect and Principal Consultant for Mulesoft – one of the most widely used integration platforms globally. Today, he works alongside our product development and delivery teams to enhance the features and functionality of PortX – PortX’s integration platform for financial institutions built on the Apache Camel framework. He is a member of the Apache Camel tooling organization and frequently contributes code to the organization’s main branch.
