Introducing the Alloy Testing Suite

Trying new things can be scary. Going live with new risk policies can be too. How can you be confident your policy will function as expected and deliver the results you expect?

We often hear from our users that they are too nervous to make changes for fear of negatively impacting outcomes. This fear of change can result in stagnant, ineffective policies and poor performance. That's where the Alloy Testing Suite comes in.

We’ve enriched our existing capabilities to offer a more comprehensive testing hub that allows clients to test new policies or changes to their decisioning logic and see projected outcomes, easing the pressure of making changes and going live. This type of agility is critical in an era of growing fraud.

What kind of test should you run?

At a high level, you can think of four categories of testing: application testing, backtesting, A/B testing, and shadow testing:

• Application testing: test your application end-to-end from the point of form submission to application creation to ensure your policy is properly configured and functioning as expected prior to go live. Conduct testing in a sandbox or mixed mode environment.

• Backtesting involves taking real, historical data and running it through production to see the results you can expect to see prior to going live.

• A/B testing lets you test different versions of risk policies against each other to determine which performs better.

• Shadow testing is one of our stand-out features in the Testing Suite. It allows clients to test a new policy in the background in parallel against the policy which is already live. Clients can use cached data, or if they wish to try out new data sources, make a real production call to those data services to get an accurate picture of the expected outcomes.

By using the right type of testing, you can gain insights that will help you make informed decisions and optimize your policies.

Why is it important to continuously iterate and test?

The importance of testing cannot be overstated. Whether you’re about to launch your risk policies for the first time or want to make changes to your existing policies, testing your risk rules is critical to identifying problems quickly and validating your projected outcomes.

Depending on your end goal, you can use testing to improve automatic good approval rates, reduce manual reviews, re-open funnels after a fraud attack, and more! Three common use cases for testing we hear from our clients are:

1) Responding more agilely to fraud threats

2) Improving the end-user experience by eliminating points of friction and

3) Testing alternative credit data sources for cash flow underwriting

Stay ahead of fraud attacks

With the rise of fraud and other forms of digital threats, it's essential to stay on top of your risk policies and be nimble when responding to emerging threats. If you do find yourself under a fraud attack, it is critical you use that experience to inform new iterations of your policies to stop the attack quickly and ensure it doesn’t happen again.

How do you spot the vulnerability in the first place? Our Reporting & Analytics dashboard can help identify anomalies or changes in outcomes and volumes that may indicate you’re going through a fraud attack. Once you’ve identified the type of fraud you’re seeing, we recommend evaluating your current setup to find the gaps that let the fraud through in the first place.

One way to do this is by running a backtest on the applications involved in the fraud attack to see what types of rules could have/would have prevented this fraud attack. What-If Analysis, the newest addition to the Alloy Testing Suite, is designed just for that purpose. If you’re still unsure about making the change, you can re-open the funnel and set up an A/B test. This will allow you to test your changes out further with a smaller volume of applicants.

Improve the client experience by removing points of friction

Testing isn't just an anti-fraud measure — it can also be used to improve conversion and customer experiences. One key metric that our clients are interested in improving is reducing the number of manual reviews, which can be costly and lead to customer abandonment.

In this instance, we recommend introducing a step-up measure like document verification to your decisioning flow to help automate cases where additional checks are required on high-risk applicants. For clients that may be hesitant about making this change for all applicants, we offer shadow testing functionality where you test out in parallel a new flow that includes a new step-up verification data vendor to see what the impact would be for your workforce using cached data without the risk of production-level impacts. You can then switch over to the new version if it performs better than your current policy or continue to iterate using different step-up vendors through a one-time integration to the Alloy SDK until you find one that works the best.

Experiment with alternative data for credit decisioning

The lending landscape is shifting towards utilizing alternative lending models such as cashflow underwriting. Alternative lending models allow banks and fintechs to service under-served markets such as immigrants and students. However, switching to alternative models can be a risky undertaking for banks and fintechs because they’ve always used a traditional underwriting model which relies heavily on the use of credit bureau data during decisioning. To combat this fear, Alloy clients can utilize A/B testing to send a smaller volume of applicants through a flow that utilizes cashflow underwriting if they’re hesitant to make the change across the board. This enables them to safely experiment with new strategies while minimizing the impact to their clients.

Alloy Testing Suite — a better way forward

The Alloy Testing Suite is a powerful tool for policy decision-makers looking to validate that their policy works as intended. It enables them to iterate on risk strategies to continuously drive better outcomes, stay on top of market changes, and swiftly respond to fraud threats. From backtesting to a/b testing and shadow testing, the Testing Suite is designed to accommodate a wide range of testing needs and provides in-platform guidance to help users choose the best testing path for their use case.

Coupled with our Reporting & Analytics dashboard, clients can continuously observe performance, test, and deploy changes in a matter of minutes through a virtuous cycle of testing and optimization. The configurability of the Alloy platform provides clients the ultimate flexibility to test in a way that works for them so they can continue to make data-driven decisions.