-
Notifications
You must be signed in to change notification settings - Fork 0
fix: make E2E tests more stable #124
base: main
Are you sure you want to change the base?
Conversation
This attempts to prevent tests from failing because of timeouts
Highlighted by Playwright accessibility checks
|
I'm testing these right now, but some are failing for me. Currently working on a fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One test failed for me: “a user can log out and log in again” on Mobile Chrome. The failing step is logOut at fixtures.ts:86.
I ran the same test 10 times and got 6 passes and 4 fails. AI suggested the flaky test is a timing issue, which fits the failure snapshot, where the menu has already closed when the test looks for “Log out”.
|
Try now @hestro |
|
@johahor However, when I ran all the e2e tests three times, I got three different results: Run 1: Run 2: Run 3: There still seem to be some flaky tests... |
|
We can reduce the amount of workers even more, because the timeouts/failures are usually caused by that. @hestro could you try decrasing the number 4 to 1 and see if that fixes it?
|
Many E2E tests failed because of timeouts, likely caused by running 8 workers running browsers at the same time.
This has now been decreased to 4.
Some tests also failed because we have a different menu in the header on mobiles now. This has been fixed by opening the menu and checking its content if we're testing on mobile screen sizes.
Furthermore, some smaller fixes highlighted by the accessibility checks from Axe have been implemented, and a slight refactoring of the E2E tests has been done.
All of these pass for me locally now, even though one must wait a little longer for the results. Please let me know if everything looks good for you guys as well.
Closes #120