Feature Flag Migration Playbook#95
Conversation
- Updated docs/what-is-mixpanel.mdx - Created images/FeatureFlagSetup.png - Created images/ExperimentSetup.png - Created images/ExperimentSetupMetrics.png - Created guides/strategic-playbooks/feature-flag-migration-playbook.mdx - Updated docs.json Mintlify-Source: dashboard-editor
kurbycchua
left a comment
There was a problem hiding this comment.
Overall comment is we need to make sure we fact check these as we do not want a customer ending up in messing up their experiment / FF data in mixpanel. Migrating the exposure of $experiment_started I find it the easy part, the challenging part is the Experiment report and the FF report of which underlying drives the analysis and the gates
| - **One-off**: From the Console, open the experiment's Pulse Results page and click _Export Report_ to download exposures and raw data as CSV. | ||
| - **Bulk / ongoing**: Use a Data Warehouse Export or the generic Event Webhook to stream raw exposure events out. | ||
|
|
||
| #### E**quivalent events** |
There was a problem hiding this comment.
Assume this section was validated by someone who is familiar with these 3rd party tools?
There was a problem hiding this comment.
this was initially written for Statsig, but as it is now vendor agnostic i will update with a generic instruction on how to export the events.
|
|
||
| Most vendors have a dedicated exposure event you can export. Each event should have the following at minimum: | ||
|
|
||
| - The enrollment ID: This is either the user ID, device ID, or group ID |
There was a problem hiding this comment.
so these "minimum" required fields we should somewhat map them to the section on "
Import events into Mixpanel as $experiment_started" where we have the source code example.. maybe something like this as comments
{
"event": "$experiment_started",
"properties": {
"distinct_id": "alice@example.com", //The enrollment ID
"time": 1709275888, //Timestamp
"$insert_id": "statsig_exp_4821_alice_1709275888", //prevent re-run duplicates
"Experiment name": "checkout_flow_test", //experiment or feature flag name
"Variant name": "treatment" //assigned variant
}
}
There was a problem hiding this comment.
there is an example code snippet in section 1b below, will incorporate the comments there
|
|
||
| This section moves the configurations themselves into Mixpanel. It covers both experiment **setup** and experiment **reporting**. | ||
|
|
||
| ### Export configurations from your vendor |
There was a problem hiding this comment.
Are we talking about completed experiments here or ongoing experiments or both? Reason I'm asking especially for Experiments I don't see like start date or sample sizes in the example? SO not sure did someone actually checked these examples to make sure that
- For historical completed experiments, or the various feature flag options they are correct in that it would recalculate any necessary metrics / exposures? Numbers might be different perhaps so that's a caveat
- For ongoing experiments - this is more challenging how do you reconcile the numbers between the 2 systems
| } | ||
| ``` | ||
|
|
||
| ## Step 3. Continue setup in the Mixpanel UI |
There was a problem hiding this comment.
ok now I'm kinda confused... so in Step 2 we use API to create experiments and now in Step 3 we are recreating again?
There was a problem hiding this comment.
unfortunately yes because there are configs in vendors that cannot be forwarded over to mixpanel eg the rollout group definition, runtime event, which are based on the events in the vendor itself
There was a problem hiding this comment.
in that case would it be better to now use the API? just a thought
|
|
||
| In the case of live experiments, migration might result in users getting reassigned to a different variant mid-experiment. We recommend completing any existing experiments and importing the historical exposure events into Mixpanel, then launching the equivalent experiment fresh in Mixpanel post-cutover. | ||
|
|
||
| 1. **If your code uses OpenFeature**: Swap the provider — replace your current provider with the Mixpanel provider and leave your existing evaluation calls unchanged, as long as your flag keys carried over identically. |
There was a problem hiding this comment.
Did we fact check these?
There was a problem hiding this comment.
do you mean the live experiment or the openfeature? for Live experiment, i checked with Shalina as she worked on that for tiket, and they did it by concluding their experiments before porting over
There was a problem hiding this comment.
Sorry to be clear when I said Did we fact check these? it's basically the whole "Step 4. Recreate critical flags and experiments in your code" section
There was a problem hiding this comment.
Like are we sure about that OpenFeature just "swap the provider"? - Not very clear on that and for the 2nd point on simply replacing the evaluation calls... just making sure these are correct code snippets as I didn't vett through them.
Summary
This update introduces a new guide for migrating feature flags and experiments from other platforms to Mixpanel. It outlines a five-part process to ensure a smooth transition, covering exposure events, configurations, and verification.
Changes
guides/strategic-playbooks/feature-flag-migration-playbook.mdx.images/FeatureFlagSetup.png,images/ExperimentSetup.png, andimages/ExperimentSetupMetrics.png.docs.jsonnavigation structure.Mintlify
0 threads from 0 users in Mintlify