About the company

Payoneer, Inc. is a leading global payment platform which powers growth for millions of small businesses and professionals as well as enterprises such as eBay, Amazon, and Walmart. The company delivers a suite of services that include cross-border payments, working capital, tax solutions, risk management and payment orchestration to more than 5 million customers worldwide.

Overview

Payoneer uses our solution to ingest and centralize logs for over 1,000 services running in production, and leverages the technology to aggregate and convert the logs to metric data which can easily be monitored for anomalous behavior.

Together with Anodot and PagerDuty, the team has a complete view of their production systems, fewer false positives for production errors, and a more streamlined troubleshooting process.

The Challenge

Before switching to our platform, Payoneer didn’t have a logging solution that could support their scaling requirements. As their systems grew, they needed a more powerful approach that would combine log data from hundreds of services and allow them to perform more dynamic calls to the system.

“When we first started to build our monitoring capabilities, we were looking at text in the logs. We were looking for specific buzzwords in certain logs, such as ‘Exception in abc’ or ‘Errors in xyz’ and so on. It was pretty cumbersome. And then we thought, how can we do better?”

The team was also relying heavily on static thresholds which were used to trigger incidents via PagerDuty. Using this strategy, the team was overwhelmed with false positives which were unavoidable due to the dynamic nature of their services and user behavior.

The Solution

With our innovation, Payoneer’s systems grew from hundreds to more than a thousand services. As the log data is ingested, the individual logs are analyzed and grouped into templates which can be used to generate metrics such as error volumes, rates, and more.

“Instead of looking at hundreds of logs, they help us to focus on the log itself that creates the noise.”

Once the data is ingested and analyzed, Payoneer sends the resulting metric data to Anodot for autonomous anomaly detection and a deeper understanding of business impact.

How the Platform Integrates to the Daily Workflow

The platform is ingesting and analyzing all of Payoneer’s log data. Using the Loggregation and Logs2Metrics features, Payoneer can pinpoint noisy logs that need to be investigated by producing metrics around them.

These metrics are then sent to Anodot, the autonomous business monitoring tool, which is then able to run their ML model and add business insights. When Anodot’s system detects anomalous behavior in the metrics, a PagerDuty alert is triggered. With this combination of tools in place, Payoneer has achieved a near-zero false positive rate.
Within the alert, there’s an option to click directly into the platform to see and investigate the errors. This also enables issues to be easily forwarded to relevant members of the team.

Overall, this solution has not only supported massive growth, it’s also improved quality of life for engineers at Payoneer who are no longer being bombarded by alerts and have a faster and easier way to investigate them.

“It also helped the developers to be more confident when a new thing is being released to production, because they know that there is something there monitoring its behavior. They trust it and they know that they’re being covered.”

“Instead of looking at hundreds of logs, the platform helps us to focus on the log itself that creates the noise.”

Yuval Molnar – Sr. Director of Production Services at Payoneer, Inc.

Summary

The team is using the platform to collect and aggregate all log data from more than 1,000 production services for more advanced anomaly detection. By reducing log data to patterned templates and metric data, Payoneer can more accurately track anomalous behavior in their applications.

More to explore

Puma

PUMA is one of the world’s leading sport-lifestyle companies that designs and develops footwear, apparel, and accessories.
Read more >

AGS

Using the platform, AGS was able to reduce their error ratio to almost zero and provide a better experience to their customers. Overall, this contributed…
Read more >

Monday.com

Up until recently the R&D team at monday.com was searching for a better way to see the full picture when it came to internal data…
Read more >