Dom/rpa_vision_v3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a7de6a488b8269927a6eb78adea3c9632acc3f33
rpa_vision_v3/.kiro/specs/visual-workflow-builder-frontend-v2/requirements.md
Dom a7de6a488b feat: replay E2E fonctionnel — 25/25 actions, 0 retries, SomEngine via serveur 
Validé sur PC Windows (DESKTOP-58D5CAC, 2560x1600) :
- 8 clics résolus visuellement (1 anchor_template, 1 som_text_match, 6 som_vlm)
- Score moyen 0.75, temps moyen 1.6s
- Texte tapé correctement (bonjour, test word, date, email)
- 0 retries, 2 actions non vérifiées (OK)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 14:04:41 +02:00

12 KiB
Raw Blame History

Requirements Document

Introduction

This document defines the requirements for the complete reconstruction of the Visual Workflow Builder (VWB) frontend. The project aims to create a modern, intuitive, and accessible user interface for creating RPA automation workflows, building on the existing working backend.

The system will provide a French-language interface for creating visual workflows using drag-and-drop interactions, with vision-based element selection and real-time execution feedback.

Glossary

  • VWB: Visual Workflow Builder - The visual workflow editor
  • Canvas: Main workspace where users build their workflows
  • Step: Action element in a workflow (equivalent to "Node" in English)
  • Connection: Link between two steps defining execution order (equivalent to "Edge")
  • Palette: Toolbox containing available step types
  • Properties_Panel: Interface for configuring step parameters
  • Workflow: Automation scenario composed of connected steps
  • Backend_VWB: Existing Flask API that handles workflow persistence and execution
  • Visual_Selector: Element selection interface based on vision and embeddings
  • ScreenCapturer: Existing screen capture service from RPA Vision V3 system
  • Visual_Embedding: Vector representation of a visual element for recognition
  • Frontend_VWB: The React-based user interface application
  • Validator: Component responsible for workflow validation and error checking
  • Executor: Component responsible for workflow execution and status tracking
  • Workflow_Manager: Component responsible for workflow persistence operations
  • Variable_Manager: Component responsible for workflow variable management
  • Interface: The complete user interface system

Requirements

Requirement 1: Modern Frontend Architecture

User Story: As a developer, I want a modern and maintainable frontend architecture, so that I can easily develop and maintain the application.

Acceptance Criteria

  1. THE Frontend_VWB SHALL use React 18+ with TypeScript for development
  2. THE Frontend_VWB SHALL use Material-UI v6+ as the primary design system
  3. THE Frontend_VWB SHALL use @xyflow/react v12+ for visual workflow rendering
  4. THE Frontend_VWB SHALL implement an architecture based on reusable components
  5. THE Frontend_VWB SHALL use Redux Toolkit for global state management
  6. THE Frontend_VWB SHALL support hot-reload for development

Requirement 2: Main Canvas Interface

User Story: As a user, I want an intuitive visual workspace, so that I can create my workflows through drag-and-drop.

Acceptance Criteria

  1. WHEN the user opens the application, THE Canvas SHALL display an empty workspace with alignment grid
  2. WHEN the user drags a step from the palette, THE Canvas SHALL allow dropping it at the desired position
  3. WHEN the user clicks on a step, THE Canvas SHALL select it visually
  4. WHEN the user drags a step, THE Canvas SHALL move it in real-time
  5. WHEN the user connects two steps, THE Canvas SHALL create a visual connection
  6. THE Canvas SHALL display a minimap for navigation in large workflows

Requirement 3: French Step Palette

User Story: As a French user, I want a toolbox with clear action names in French, so that I can easily understand the available functionalities.

Acceptance Criteria

  1. THE Palette SHALL organize steps in French categories (Web Actions, Logic, Data, Control)
  2. WHEN the user hovers over a step, THE Palette SHALL display an explanatory tooltip in French
  3. WHEN the user types in the search, THE Palette SHALL filter steps by French name
  4. THE Palette SHALL use intuitive icons for each step type
  5. THE Palette SHALL allow drag-and-drop to the Canvas

Requirement 4: Step Configuration

User Story: As a user, I want to easily configure the parameters of each step, so that I can customize their behavior.

Acceptance Criteria

  1. WHEN the user selects a step, THE Properties_Panel SHALL display its configurable parameters
  2. WHEN the user modifies a parameter, THE Properties_Panel SHALL validate the input in real-time
  3. WHEN a required parameter is missing, THE Properties_Panel SHALL display an error indicator
  4. THE Properties_Panel SHALL adapt the interface according to parameter type (text, number, list, boolean)
  5. THE Properties_Panel SHALL allow screen element selection for interaction steps

Requirement 5: Vision-Based Visual Element Selector

User Story: As a user, I want to select elements purely visually on a screenshot, so that I can easily configure interaction steps without using CSS or XPath selectors.

Acceptance Criteria

  1. WHEN the user clicks "Select an element", THE Visual_Selector SHALL capture the screen via the existing ScreenCapturer API
  2. WHEN the user hovers over an area of the capture, THE Visual_Selector SHALL display a preview of the selectable area
  3. WHEN the user clicks on an area, THE Visual_Selector SHALL create a visual embedding of that area
  4. WHEN the selection is confirmed, THE Visual_Selector SHALL store the embedding and reference image for the step
  5. THE Visual_Selector SHALL allow visualization of the selected element with its visual context

Requirement 6: Variable Management

User Story: As a user, I want to create and use variables in my workflows, so that I can make my automations flexible and reusable.

Acceptance Criteria

  1. THE Variable_Manager SHALL allow creating, modifying and deleting variables
  2. WHEN the user creates a variable, THE Variable_Manager SHALL validate name uniqueness
  3. WHEN the user types ${variable_name}, THE Properties_Panel SHALL provide autocompletion
  4. THE Variable_Manager SHALL support different types (text, number, boolean, list)
  5. THE Variable_Manager SHALL allow defining default values

Requirement 7: Validation and Visual Feedback

User Story: As a user, I want to be notified of errors in my workflow, so that I can correct problems before execution.

Acceptance Criteria

  1. WHEN a step has missing parameters, THE Validator SHALL display a red indicator on the step
  2. WHEN the workflow has disconnected steps, THE Validator SHALL highlight them in orange
  3. WHEN the workflow contains cycles, THE Validator SHALL display a warning
  4. THE Validator SHALL prevent execution if critical errors exist
  5. THE Validator SHALL display an error summary in a dedicated panel

Requirement 8: Real-Time Execution and Feedback

User Story: As a user, I want to see the execution state of my workflow in real-time, so that I can understand what is happening and identify problems.

Acceptance Criteria

  1. WHEN the user starts execution, THE Executor SHALL send the workflow to Backend_VWB
  2. WHEN a step executes, THE Canvas SHALL highlight it in blue
  3. WHEN a step succeeds, THE Canvas SHALL display it in green
  4. WHEN a step fails, THE Canvas SHALL display it in red with the error message
  5. THE Executor SHALL display an execution summary (duration, success rate)

Requirement 9: Save and Load Operations

User Story: As a user, I want to save my workflows and reload them later, so that I don't lose my work.

Acceptance Criteria

  1. WHEN the user clicks "Save", THE Workflow_Manager SHALL send the data to Backend_VWB
  2. WHEN the user opens an existing workflow, THE Workflow_Manager SHALL load it from Backend_VWB
  3. THE Workflow_Manager SHALL display the list of available workflows
  4. THE Workflow_Manager SHALL allow renaming and deleting workflows
  5. THE Workflow_Manager SHALL handle save conflicts (multiple versions)

Requirement 10: French Internationalization

User Story: As a French user, I want an interface entirely in French with accessible terminology, so that I can easily understand all functionalities.

Acceptance Criteria

  1. THE Interface SHALL use exclusively French terminology (Step, Connection, Workspace)
  2. THE Interface SHALL provide explanatory tooltips in French for all elements
  3. THE Interface SHALL display error messages in clear and understandable French
  4. THE Interface SHALL use universal icons complemented by French text
  5. THE Interface SHALL include an accessible glossary of technical terms

Requirement 11: Accessibility and Ergonomics

User Story: As a user, I want an accessible and ergonomic interface, so that I can use the application efficiently even without technical expertise.

Acceptance Criteria

  1. THE Interface SHALL support complete keyboard navigation
  2. THE Interface SHALL comply with WCAG 2.1 level AA standards
  3. THE Interface SHALL provide keyboard shortcuts for frequent actions
  4. THE Interface SHALL adapt element sizes for different screen resolutions
  5. THE Interface SHALL include a contextual help mode for new users

Requirement 12: Performance and Responsiveness

User Story: As a user, I want a fluid and responsive interface, so that I can work efficiently even with complex workflows.

Acceptance Criteria

  1. THE Interface SHALL maintain 60fps when moving steps
  2. THE Interface SHALL load a workflow of 100 steps in less than 2 seconds
  3. THE Interface SHALL use virtualization for long lists
  4. THE Interface SHALL implement debouncing for expensive operations
  5. THE Interface SHALL optimize rendering to avoid unnecessary re-renders

Requirement 13: Backend Integration

User Story: As a system, I want seamless integration with the existing backend, so that I can ensure data and functionality consistency.

Acceptance Criteria

  1. THE Frontend_VWB SHALL use the Backend_VWB REST API for all CRUD operations
  2. THE Frontend_VWB SHALL handle backend communication errors gracefully
  3. THE Frontend_VWB SHALL implement a retry system for failed requests
  4. THE Frontend_VWB SHALL validate data client-side before sending to backend
  5. THE Frontend_VWB SHALL maintain data format consistency with the backend

Requirement 14: Vision-Based Testing and Quality

User Story: As a developer, I want complete test coverage using only real visual approaches, so that I can ensure reliability without depending on synthetic captures.

Acceptance Criteria

  1. THE Frontend_VWB SHALL have at least 80% unit test coverage for business logic
  2. THE Frontend_VWB SHALL include integration tests using only real screenshots
  3. THE Frontend_VWB SHALL use property-based tests for visual embedding validation
  4. THE Frontend_VWB SHALL pass all TypeScript and ESLint linting tests
  5. THE Frontend_VWB SHALL test integration with existing ScreenCapturer API without synthetic data

Requirement 15: Interactive Documentation Tabs

User Story: As a user, I want accessible documentation for each tool and module directly in the interface, so that I can understand how to use each feature without leaving the application.

Acceptance Criteria

  1. THE Interface SHALL include a documentation tab for each major tool and module
  2. WHEN the user opens a documentation tab, THE Interface SHALL display contextual help for that specific tool
  3. THE Documentation SHALL include step-by-step guides with visual examples for each feature
  4. THE Documentation SHALL be available in French with clear terminology
  5. THE Documentation SHALL include interactive examples that users can try directly

Requirement 16: Documentation and Maintenance

User Story: As a developer, I want complete and up-to-date documentation, so that I can easily maintain and evolve the application.

Acceptance Criteria

  1. THE Documentation SHALL be centralized in the docs/ directory of the project
  2. THE Documentation SHALL include a development guide with architecture and conventions
  3. THE Documentation SHALL include a user guide with screenshots
  4. THE Code SHALL include comments in French for complex parts
  5. THE Documentation SHALL be automatically updated when API changes occur
Reference in New Issue View Git Blame Copy Permalink