Python-bloggers

A Guide to Validating R (Shiny) Applications in Pharma

This article was first published on Appsilon | Enterprise R Shiny Dashboards , and kindly contributed to python-bloggers. (You can report issue about the content on this page here)
Want to share your content on python-bloggers? click here.

Validating R Shiny applications is critical in the pharmaceutical industry, where data integrity, compliance, and patient safety are essential. Building on our post on R package validation, Shiny apps introduce unique challenges due to their interactive nature and real-time data processing compared to packages. To meet FDA (US), EMA (EU), NMPA (China), and other regulatory standards standards, these apps must undergo rigorous GxP validation to ensure they function reliably and securely.

Curious how to make GxP compliance easier from the start? Check out our post on why it all begins with the Definition of Done.

As the shift towards open-source tools like R continues, validating both packages and interactive apps is increasingly important. Frameworks like Rhino and Teal support this process. These tools help ensure Shiny apps maintain reproducibility, performance, and usability standards in clinical settings.

In this post, we’ll explore the challenges of validating Shiny apps in pharma, as well as the guidelines, best practices, and tools that can streamline the validation process.

What is R App Validation and Why Does It Matter? 

R app validation is the process of establishing documented evidence that an R-based application, such as a Shiny app, consistently performs its intended functions accurately, reliably, and in compliance with regulatory standards. According to GxP (“Good Practice”) guidelines — validation must demonstrate a high degree of assurance that the application produces results that meet predetermined specifications under all expected conditions.

In the pharmaceutical context, validation ensures that software tools used in clinical trials, drug development, and regulatory submissions maintain data integrity, meet compliance requirements, and ultimately protect patient safety. Unlike static R packages, Shiny apps are dynamic, involving real-time data inputs, user interactions, and visualizations, which adds complexity to the validation process.

Unique Challenges of Validating Interactive Applications like Shiny Apps

Validating R Shiny applications presents unique challenges that differ significantly from those faced when validating traditional scripts or batch processes. These challenges include:

Dynamic Interactions

Unlike static scripts, Shiny apps are highly interactive and reactive. They must handle real-time data inputs and provide instant outputs, making validation more complex. Every possible user interaction and data input scenario needs to be considered during the validation process.

User Behavior Variability

Shiny apps are often used by a diverse group of users, ranging from data scientists to clinicians with varying levels of technical expertise. The app must function correctly regardless of how it is used — whether it’s handling complex queries from an advanced user or navigating simple tasks by a less experienced user. This requires extensive user acceptance testing (UAT) [testing in the real world, sometimes called beta testing] to ensure that the app meets usability standards for all possible user types.

User Acceptance Testing

User Access and Security

In many cases, Shiny apps require managing different access levels for various user groups. Some users might need restricted access to certain features or datasets based on their roles.

Testing for unauthorized access is crucial to ensure that sensitive data remains secure and users only have access to the functions they are permitted to use. This involves rigorous role-based testing and security checks to ensure that unauthorized users cannot gain access to restricted areas of the app.

Want to keep your Shiny apps secure? Dive into our guide on making R Shiny apps safer.

Performance Under Load

Shiny apps frequently handle large datasets and multiple concurrent users. They must perform reliably under varying loads — from a single user analyzing a small dataset to a scenario where many users are simultaneously querying large datasets. This requires rigorous performance testing, including load testing to simulate multiple users and stress testing to identify potential bottlenecks or failures under extreme conditions.

Cross-Platform Compatibility

Shiny apps must perform consistently across various browsers, devices, and operating systems. Ensuring this cross-platform compatibility requires thorough testing to identify and resolve any bugs or inconsistencies that could affect user experience or data accuracy.

Tools like shinylive and webR are particularly valuable for this purpose: shinylive allows Shiny apps to run directly in web browsers without relying on an R server, simplifying testing across different environments and reducing dependencies. Meanwhile, webR enables R code to execute in the browser using WebAssembly, ensuring reliable functionality on any platform. Together, these tools help maintain consistent performance and usability across all platforms.

“The promise of WebAssembly with webR is that researchers can produce analytics with R & Shiny, … and be as confident as is possible that any user will be able to get it up and running quickly on their browser. And it will be run in a decades’ time exactly the same way as it is today. …”
George Stagg, webR maintainer and engineer at Posit PBC.

Regulatory Requirements for Validation

To operate within regulated environments, Shiny apps must comply with several key regulatory requirements. Validation principles will remain consistent across regulatory agencies, although details on the process or specific documentation might differ. Let’s see some examples of those principles and regulatory standards:

GxP Validation

GxP guidelines, which include Good Clinical Practice (GCP), Good Laboratory Practice (GLP), and Good Manufacturing Practice (GMP), set the standards for the development and maintenance of software in regulated industries.

For Shiny apps, GxP validation involves ensuring the app performs as intended in all situations, with documented evidence of all validation activities. This includes a comprehensive risk assessment, detailed testing plans, traceability matrices to map requirements to tests, and change management processes to handle updates or modifications.

FDA Guidelines

The FDA’s “General Principles of Software Validation” provides specific guidelines on how to validate software used in clinical trials or for regulatory submissions. It emphasizes that software must be validated to produce accurate, reliable, and consistent results, with thorough documentation for each stage of the validation process. The FDA also focuses on risk management, requiring that all potential risks to patient safety or data integrity be identified and mitigated during validation.

EMA Guidelines

The EMA outlines similar requirements, particularly through the Good Automated Manufacturing Practice (GAMP) guidelines.

GAMP 5 V-Model

GAMP focuses on ensuring that all computerized systems used in regulated activities — including Shiny apps — are validated to perform reliably and securely. It mandates strict controls over data security, access, and audit trails to ensure data integrity and compliance with regulatory standards.

Compliance with these regulatory requirements is essential for any Shiny app used in pharmaceutical contexts. Failure to validate according to these standards can lead to severe consequences, including financial penalties, regulatory sanctions, or compromised patient safety.

Key Validation Criteria for Shiny Applications

Reproducibility
Ensuring that Shiny applications deliver consistent results across different environments is crucial for regulatory compliance. Unlike static R packages, Shiny apps must also account for user interactions and real-time data processing. This requires validating that the app produces the same outputs given the same inputs, regardless of factors like operating system, browser type, device, or user behavior.

Validation should cover all dynamic elements, such as input fields, reactive expressions, and visualizations, confirming they function consistently under various scenarios and environments.

Functionality
The core of any validation process is ensuring that the app performs its intended functions correctly and reliably. This means that all components of the app, including input fields, data processing scripts,  output visualizations, or generated reports, should work as expected without errors. Each feature should be tested to confirm it operates as intended under a range of scenarios.

Error handling in Shiny Applications is usually more complex and challenging than in R Packages. That is because we should handle errors at different levels (user actions, input data, other software) and in a growing number of scenarios. Therefore, It is important to design your app carefully so it is prepared to react in all situations (wrong input data, wrong input parameters, API not available …) quickly and user-friendly.

Performance
The app must perform efficiently under different conditions, such as varying data sizes or multiple simultaneous users. Performance validation involves stress-testing the app to identify bottlenecks or resource constraints and ensure it meets predefined performance standards.

Data Security and Privacy

Applications handling sensitive clinical data must comply with data protection regulations like GDPR or HIPAA. This involves validating that the app employs adequate security measures, such as encryption, access controls, and data anonymization, to protect patient data and maintain privacy.

This is a topic that, again, is more challenging with Shiny Apps than with packages. Developers should consider app security from the beginning of the app design. It is important to always give only the necessary access to sensitive data, to be able to switch environments easily, and be careful with external software.

Software-Based Validation for Shiny Apps

This approach treats Shiny apps like any other software product, emphasizing a structured Software Development Life Cycle (SDLC), automated testing, Continuous Integration/Continuous Deployment (CI/CD), comprehensive documentation, and modular design.

SDLC

For Shiny apps, this means not only validating individual components and functions but also shiny modules, UI components, and outputs and ensuring the app performs reliably during real-time data processing and user interactions. Automated testing tools specific to Shiny, like end-to-end tests and shinyValidator, should be used to cover all dynamic elements, while CI/CD pipelines ensure that any changes are continuously tested and validated.

Guidelines for Validating R Shiny Applications in Pharma

Here’s a structured guide to help you navigate the validation process, anticipate common challenges, and select the right tools.

Step 1: Define Validation Requirements

Step 2: Plan and Design the Validation Process

Step 3: Implement Automated Testing

Step 4: Conduct Performance Testing

Step 5: Perform User Acceptance Testing (UAT)

Step 6: Ensure Data Security and Privacy Compliance

Shiny application security is a fundamental necessity. Learn more in “How to Make Your Shiny Apps Secure.”

Step 7: Implement Continuous Integration/Continuous Deployment (CI/CD)

Step 8: Maintain Validation Over Time

Step 9: Prepare for release for submission

Tools and Best Practices for Validating R Shiny Applications

This section provides an overview of the most important tools and best practices that can help facilitate validation, maintain compliance, and ensure high performance.

Tools for Validation

  1. Valtools: Designed specifically for clinical research, this tool automates the validation process, making it easier to meet compliance requirements.
  2. profvis: Profiles R code to identify performance bottlenecks, helping to optimize the app’s efficiency.
  3. shinyloadtest: Simulates user traffic to test the app’s performance under load, identifying potential bottlenecks and ensuring stability under high demand.
  4. rsconnect: Deploys Shiny apps and monitors performance metrics over time, providing valuable insights into app performance in production environments.
  5. rhino: A framework for building enterprise-grade Shiny apps using a modular design that improves maintainability and scalability. It integrates with CI/CD tools to automate testing and deployment.
  6. teal: Creates interactive data exploration apps for clinical trials with features like real-time data filtering, code reproducibility, and regulatory-ready reporting. It integrates with pharmaverse packages like {admiral} and {rtables} to support clinical data analysis and reporting.
  7. shinymeta: Facilitates reproducible reporting by exposing the underlying R code behind reactive expressions, enhancing transparency in regulatory settings.
  8. GitHub Actions / Jenkins: Automate testing and deployment processes, ensuring that every change to the app is validated before going live.
  9. R Markdown / Quarto: Document all validation activities, including test plans, results, and change logs. These tools provide flexible and accessible formats for generating validation reports required for audits and inspections.

A Successful FDA Submission Using R

The R Consortium’s Submission Working Group hit an important milestone with Pilot 4, showcasing how open-source tools like R and new technologies—WebAssembly and containers—can streamline FDA submissions. This effort is all about updating the way clinical trial data is submitted, making the process more efficient and transparent.

The Process

Pilot 4 was all about testing if a Shiny app could be packaged and submitted using WebAssembly and containers. Here’s how the team made it happen:

Tools Used

The Outcomes

Pilot 4 proved that it’s possible to submit a fully functioning Shiny app using these cutting-edge technologies. It showed that:

Conclusion

Pilot 4 highlights how open-source technologies can improve FDA submissions, making the process faster, clearer, and more efficient. By collaborating with industry partners and using tools like webR and containers, the team is pushing the boundaries of what’s possible in clinical trial reporting.

Read the full blog post: Testing Containers and WebAssembly in Submissions to the FDA.

Wrapping Up A Guide to Validating R Applications in Pharma 

Validating R Shiny applications in the pharmaceutical industry is crucial for maintaining data integrity, regulatory compliance, and patient safety.

As open-source tools like R continue to gain traction, the unique challenges of validating dynamic and interactive applications must be addressed with robust frameworks and best practices. By implementing automated testing, performance monitoring, and comprehensive documentation, pharmaceutical companies can ensure their applications remain reliable and secure in clinical environments.

If you need expert guidance to simplify your validation process and ensure compliance with regulatory standards, our team is here to help. Contact us today to learn how we can support your journey to achieving secure and validated R Shiny applications.

Looking to ensure your clinical software is GxP-compliant? Learn how to validate Shiny apps in our next post.

Resources and Communities

Communities

The post appeared first on appsilon.com/blog/.

To leave a comment for the author, please follow the link and comment on their blog: Appsilon | Enterprise R Shiny Dashboards .

Want to share your content on python-bloggers? click here.
Exit mobile version