diff --git a/docs/getting-started/contributing.md b/docs/getting-started/contributing.md index e62e44d0b..59d5953ca 100644 --- a/docs/getting-started/contributing.md +++ b/docs/getting-started/contributing.md @@ -4,7 +4,7 @@ title: For Contributors slug: /getting-started/for-contributors --- -The Experimenter documentation hub is managed in the [mozilla/experimenter-docs](https://github.com/mozilla/experimenter-docs) repository. You will need a GitHub account to contribute, and if you are not already in the [Project Nimbus GitHub team](https://github.com/orgs/mozilla/teams/project-nimbus/members), you may need to request write access in the #nimbus-project Slack channel. +The Experimenter documentation hub is managed in the [mozilla/experimenter-docs](https://github.com/mozilla/experimenter-docs) repository. You will need a GitHub account to contribute, and if you are not already in the [Project Nimbus GitHub team](https://github.com/orgs/mozilla/teams/project-nimbus/members), you may need to request write access in the #nimbus-dev Slack channel. :::info diff --git a/docs/messaging/desktop/desktop-messaging.md b/docs/messaging/desktop/desktop-messaging.md new file mode 100644 index 000000000..20f46230f --- /dev/null +++ b/docs/messaging/desktop/desktop-messaging.md @@ -0,0 +1,108 @@ +--- +id: desktop-messaging +title: Desktop Messaging +slug: /messaging/desktop/desktop-messaging +--- + +Guide to configuring messaging system experiments on Firefox Desktop, focusing on what is specific to messages: message configuration, behaviors, and the relevant tools and reference docs. + +## Before You Start + +Familiarize yourself with the key messaging system concepts: + +- **[Surfaces](/messaging/desktop/desktop-messaging-surfaces)** - Different message types (doorhangers, feature callouts, spotlights, etc.) +- **[Display Logic](/messaging/desktop/display-logic)** - Triggers, targeting, and frequency capping +- **[Configuration & Frequency](/messaging/desktop/message-configuration)** - Detailed configuration options + +The general experiment lifecycle is the same for messaging as for any other experiment and is documented separately: + +- **[Workflow overview](/workflow/overview)** and **[Getting Started for engineers](/getting-started/for-engineers)** - experimenter fundamentals and creating experiments +- **[Local Enrollment](/platform-guides/desktop/enroll-locally)** and **[Preview Mode](/platform-guides/desktop/preview)** - forced and natural enrollment, and previewing branches +- **[Testing](/workflow/testing)**, **[Launching](/workflow/launching)**, **[Monitoring](/workflow/monitoring)**, and **[Ending](/workflow/ending)** - testing, launch requests, and post-launch + +:::info + +Whenever you are working with the messaging system, use **#omc** for experiment review and launch requests rather than #ask-experimenter. + +::: + +### Helpful Tools & Resources + +- **[Skylight](https://fxms-skylight.netlify.app/)** - View live messaging experiments, rollouts, and dashboards +- **[ASRouter Devtools](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/debugging-docs.html)** - Preview and test message JSON +- **[nimbus-devtools Guide](/resources/nimbus-devtools-guide)** - Advanced testing and debugging +- **#omc** - Slack channel for messaging system questions and reviews + +The Firefox [Messaging System source docs](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/index.html) are the most up-to-date reference for anything specific to the messaging system: + +- **[UI Templates](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/#ui-templates)** - The different message surfaces (doorhangers, feature callouts, spotlights, etc.) and their schemas +- **[Trigger Listeners](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/TriggerActionSchemas/index.html)** - The user actions that can cause a message to be shown +- **[Targeting Attributes](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/targeting-attributes.html)** - The client and browser state you can target against +- **[Frequency Caps & Groups](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/frequency-caps.html#message-groups)** - How often a message can be shown, and how to cap groups of messages together +- **[Schemas](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/index.html)** - Message schema definitions + +## Configuring Message Branches + +Each treatment branch holds the message JSON that matches the brief's design, content, and targeting. When configuring branches: + +- Set the appropriate [feature configuration](/messaging/desktop/message-configuration) for the surface you're using +- Ensure the JSON matches the [message schema](/messaging/desktop/message-configuration) +- For multi-branch experiments, give the control branch a real `message_id` rather than an empty object so that it records an exposure event + +:::warning + +An empty object for control will never trigger or go through ASRouter and won't record an exposure event. As a result, the exposure analysis on Experimenter won't work correctly as there's no baseline to compare the treatments to. + +::: + +Ask in #omc for review if you need clarification on multi-branch experiment setup. + +## Selecting a Feature ID + +Which feature ID a messaging experiment uses depends on whether it's a holdback/rollout or a standard experiment, and on which surface the message uses. + +### Holdback Experiments & Rollouts + +For rollouts and holdback experiments, you should pick up a placeholder Feature ID (`fxms-message`) that is not in use. See the [list of current messaging system feature IDs](https://experimenter.services.mozilla.com/nimbus/?status=Live). You can verify what is in use by checking the Experimenter UI and filtering for the specific feature ID using the **All Features** dropdown on the left. + +### Non-Holdback Experiments + +When picking a feature ID for a non-holdback experiment, you should first prefer to run it on the [feature that matches the surface](/messaging/desktop/desktop-messaging-surfaces) that your experiment is using (`spotlight`, `featureCallout`, `infobar`, `cfr`, etc.). + +If there are any other experiments using that feature ID that overlap in time with yours, you may still be able to use it, but ask in #omc to verify. + +**Audience Overlap Considerations:** + +In the case of experiment targeting overlaps you must ensure that the sum of all audience sizes, including the audience size of your experiment, is no more than 100% for each segment that overlaps. For example, if an experiment exists that uses the same targeting segment as yours is enrolling at 30%, your experiment may use that same feature ID at up to 70%, for a combined 100% of the population of that targeting segment. + +:::note + +Co-enrollment is supported for `fxms-message` feature IDs as of Firefox 152, so these audience-overlap constraints may not apply when targeting Firefox 152+. + +::: + +If you are unable to use the feature that matches the surface of your message, reach out to #omc or a data scientist to discuss other options. + +**Rollout Priority:** + +Experiments take priority over rollouts, so one option is to share an ID with an existing rollout. If all IDs are taken by an experiment, it may still be usable depending on audience overlap. Check whether the targeting populations intersect. If your experiment targets Germany and the existing one targets Canada, they don't share users and can safely reuse the same ID. + +The key is that the 'effective audience' (after all targeting criteria) doesn't overlap, not just that the IDs are "free." This also includes things like the channel, locales, and the audience percentage. + +## Previewing Message UI with `about:asrouter` + +`about:asrouter` (the [ASRouter Devtools](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/debugging-docs.html)) is the fastest way to iterate on a message's UI without enrolling. To enable it, set `browser.newtabpage.activity-stream.asrouter.devtoolsEnabled` to `true` in `about:config`, then open `about:asrouter`. + +Paste your message JSON into the editor to render it immediately and check the layout, copy, and buttons. You can also inspect the messages currently available to the client and the provider they came from, and reset a message's impressions and frequency-cap history (or block and unblock it) so you can re-test as if you had never seen it. + +:::info + +`about:asrouter` previews only the **UI** of a message. It does **not** evaluate targeting or triggering. To test the full behavior of a message, enroll in the experiment. See [Local Enrollment](/platform-guides/desktop/enroll-locally) and [Preview Mode](/platform-guides/desktop/preview). + +::: + +## Additional Resources + +- [Messaging System Documentation](/messaging/overview) +- [Experimenter Documentation](https://experimenter.info) +- [nimbus-devtools Guide](/resources/nimbus-devtools-guide) diff --git a/docs/messaging/desktop/display-logic.md b/docs/messaging/desktop/display-logic.md index 8d885b64d..15d2b28e2 100644 --- a/docs/messaging/desktop/display-logic.md +++ b/docs/messaging/desktop/display-logic.md @@ -11,10 +11,22 @@ There are three types of display logic you should consider when using any of the * **Targeting**: For what kinds of users, or browser state, should the message be shown? (e.g. locale, country, prefs) * **Frequency and Blocking**: How many times should the message be shown, and in what conditions should it never be shown again? +:::note + +The [Messaging System source docs](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/index.html) are the most up-to-date reference for [triggers](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/TriggerActionSchemas/index.html) and [targeting attributes](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/targeting-attributes.html). The examples below cover the most common cases. Check the source docs for the complete list and exact names. + +::: + ## Triggers A trigger is a particular "event" (or set of events) that must occur for the message to be displayed. You can see a [complete list of triggers](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/TriggerActionSchemas/index.html#available-trigger-actions) in Firefox source docs, but here are some of the most common ones: +:::note + +Always check the [Trigger Action Schemas source docs](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/TriggerActionSchemas/index.html) for the most up-to-date list of triggers and the exact trigger context each one provides. + +::: + ### `openURL` The message will display when the user navigates to a URL that matches the provided hosts or patterns. The most general configuration for this is `"*"`, which means the message will effectively trigger the first time any window is opened. @@ -23,6 +35,23 @@ The message will display when the user navigates to a URL that matches the provi The message will display when a preference has changed. Note that this does *not* trigger if the preference was already set to the desired value at startup. +### `defaultBrowserCheck` + +The message will display when Firefox checks whether it is the default browser. This check runs both at startup and when a new tab is opened, so the trigger supplies a `source` value to tell the two apart: + +* `source == 'startup'`: the check ran during browser startup +* `source == 'newtab'`: the check ran because the user opened a new tab + +A common pattern is to gate a default-browser message to new tabs only by combining the trigger with `source == 'newtab'` in your targeting. Pairing `defaultBrowserCheck` with `source == 'newtab'` is appropriate when you want the message to appear on the new tab page; use the new-tab feature-callout trigger (below) instead when you are rendering a feature callout anchored to new-tab UI. See the [complete trigger list](https://firefox-source-docs.mozilla.org/toolkit/components/messaging-system/docs/TriggerActionSchemas/index.html#available-trigger-actions) in Firefox source docs for the exact trigger names and the `source` values each one provides. + +:::warning + +You may notice `source == 'newtab'` appears to "fail" in the JEXL debugger. This is because `source` is part of the **trigger context**; it only exists when the trigger actually fires. The standalone JEXL debuggers (`about:asrouter` and nimbus-devtools) evaluate targeting against your **client targeting context**, which does *not* include trigger-supplied variables like `source`. + +As a result, an expression referencing `source` evaluates against an unknown variable and returns an [empty result](/platform-guides/desktop/targeting-debug#targeting) (neither `true` nor `false`) in the debugger, even though it will work correctly at runtime when the trigger fires. This is expected: targeting that depends on trigger context must be verified through [force or natural enrollment](/messaging/desktop/desktop-messaging#testing-the-experiment), not the debugger. + +::: + ## Targeting @@ -36,6 +65,12 @@ You can see a complete [list of the available attributes](https://firefox-source * `region` (based on IP, e.g. `US`, `FR`) * `preferenceValue` (any Firefox preference value) +:::note + +Always check the [Targeting Attributes source docs](https://firefox-source-docs.mozilla.org/browser/components/asrouter/docs/targeting-attributes.html#attributiondata) for the most up-to-date list of targeting attributes and their exact names. + +::: + ## Frequency and Blocking diff --git a/docs/messaging/desktop/message-lifecycle.md b/docs/messaging/desktop/message-lifecycle.md index 01271e527..c5dc99af4 100644 --- a/docs/messaging/desktop/message-lifecycle.md +++ b/docs/messaging/desktop/message-lifecycle.md @@ -18,7 +18,7 @@ Once an idea has been developed, the next step is designing the experiment messa ### Running an Experiment First, determine if your experiment will require any on-train development work to support the launch as well as any translated strings that need to land prior. Once that's been determined, we can get started with creating the experiment. -For a more in-depth guide and step by step process, visit [OMC: Experimenter onboarding document](https://mozilla-hub.atlassian.net/wiki/spaces/FIREFOX/pages/233406786/OMC+Experimenter+Onboarding) +For a more in-depth guide and step by step process, visit the [Desktop Messaging Onboarding](/messaging/desktop/desktop-messaging) guide. ### Message in Firefox @@ -107,6 +107,20 @@ It is also possible to specifically include or exclude users into experiments ba Note that when using targeting or groups in this way the message **impressions** are removed once the message has been removed or the experiment has ended. ::: +:::info + +When re-launching a message, frequency caps and [blocks](/messaging/desktop/display-logic#blocking) are keyed on `message_id`. If you re-run an experiment (or convert it to a rollout) with a **new** `message_id`, users who already saw the message have no impression or block history under the new ID and may be shown it again, leading to duplicate messaging and message fatigue. + +To avoid this, reuse the same `message_id` so history carries over, and end the original experiment or rollout before launching the re-run (or ensure the audiences don't overlap). Note that because impressions are cleared once the original ends, a gap between the two can still re-expose users even with the same `message_id`. + +::: + +:::info + +As of Firefox 152, message impressions are retained for 6 months after an experiment or rollout ends, rather than being cleared immediately. For audiences on Firefox 152+, this means you no longer need to end the original experiment or rollout (or avoid a gap) before launching the re-run. Reusing the same `message_id` carries history over. Conversely, if you *want* to show the message to those users again, you must use a **new** `message_id`. + +::: + ### No Action to Block Message For the doorhanger template we had a built-in (into the message surface) button to block a message. For all other surfaces there is no way for the user to block a message from ever being shown. Usually campaigns have had a maximum of 1-2 impressions and it was not considered to add a block action. The benefit of blocking would be that we could group similar messages as part of a **"campaign"** (for example Mozilla VPN) and use the block signal as an indicator not to show future messages with similar content. diff --git a/docs/workflow/ending.md b/docs/workflow/ending.md index 95485eb29..596383151 100644 --- a/docs/workflow/ending.md +++ b/docs/workflow/ending.md @@ -41,3 +41,9 @@ The process to end the experiment is similar to that of ending enrollment: press Some notes: * Automated analyses are based on the actual observation period (period between enrollment and experiment ending), not the planned period. So experiments that remain open past their scheduled end date will have more days worth of data included in their results. + +:::info + +For messaging system experiments, ending an experiment or rollout **clears the message impression history** tied to it. If you plan to re-launch the message (for example, converting an experiment into a rollout), reuse the same `message_id` and avoid a gap between ending the original and launching the re-run, otherwise users who already saw the message may see it again. As of Firefox 152, impressions are instead retained for 6 months after ending, so for audiences on Firefox 152+ this gap no longer matters. See [Re-launching a message](/messaging/desktop/message-lifecycle#history-state-of-past-experiments) for details. + +::: diff --git a/sidebars.js b/sidebars.js index 58dfec927..44c5a6f6b 100644 --- a/sidebars.js +++ b/sidebars.js @@ -233,6 +233,7 @@ module.exports = { label: "Messaging System", items: [ "messaging/experiments-and-user-messaging", + "messaging/desktop/desktop-messaging", "messaging/desktop/messaging-surfaces", "messaging/desktop/display-logic", "messaging/desktop/message-configuration",