Bug Reproduction: From Issue to Reproducible Pack

💡 One-sentence summary: Use Grok Bot to build your AI bug reproduction pipeline — read the report, connect issue tracking and staging, reproduce by hand with a fresh test account, and produce a reproducible pack with exact steps, screenshots, and environment info, with credentials never entering the chat.

This is the first lesson of Route B (for programmers), from the same page of official roles as Performance Investigation and using the same official six-step method. This lesson corresponds to the official use-cases page, one of 8 roles: Bug reproduction (Bug Reproduction Specialist). The official positioning for it is: turning bug reports into reliable reproduction packs — not "helping you interpret an error message," but a reliable deliverable that developers can use directly for debugging. The two connection types officially named are the issue tracking system and the staging environment.

The whole lesson takes about 12 minutes. The official requirements for this role include one special rule: test credentials go through a secure handoff and never enter the chat — step 3 of this lesson covers exactly how to implement it.

What you'll be able to do after this lesson

• Build a "Bug Reproduction Specialist" Bot: connect the issue tracking system and the staging environment, and turn bug reports into reliable reproduction packs
• Have the Bot reproduce step by step in staging with a fresh test account: exact steps, expected vs. actual behavior, screenshots, browser and system info, plus a minimal test case if it can be distilled
• Test credentials go through secure handoff (Vault auto-fill / Take over for manual entry), never entering the chat and never touching production customer data
• Save the reproduction workflow as a skill to rerun by name; incoming new bug reports can be wired to an event-triggered routine (optional)

Why use Grok Bot instead of a web-based AI?

Capability	Web-based AI	Grok Bot
Can read your issues and staging	❌ Screenshots plus verbal descriptions; the environment relies entirely on what you relay	✅ Direct connection to issue tracking via connector; logs into staging itself using a cloud computer browser
Reproduces by hand instead of copying the report	❌ Can only guess the cause from the description	✅ Executes step by step in staging and captures screenshot evidence; the steps come from actually running them
Credentials don't leak	❌ Passwords have to be pasted into the chat box	✅ Vault auto-fills without exposing values; sensitive steps use Take over so you type them by hand
Remembers the reproduction standard	❌ You have to re-explain everything in every conversation	✅ The reproduction workflow is saved as a skill, rerun by name, with a consistent standard
New bugs start running automatically	❌ Every report has to be pasted manually	✅ Event-triggered routines (GitHub/Linear/Sentry/PagerDuty)

Your current predicament

• Bug reports are vague: "clicked and nothing happened" — you have to go back and forth asking about the environment, account, and steps
• Reproduction is manual: spin up a test environment, create test data, click through step by step — an afternoon is gone
• Reproduction pack quality varies: a missing step, screenshots not saved, environment info forgotten — developers have to ask another round

When to use this technique

When you need to:

• Turn a muddled bug report into a reproducible pack developers can use directly: exact steps, expected vs. actual, screenshots, environment info
• Get the "read report → enter staging → reproduce step by step → write the pack" pipeline running, with you only reviewing the pack
• Keep the reproduction process away from production customer data, with test credentials never entering the chat

And you don't want to:

• Open environments, click through steps, and take screenshots one by one yourself
• Paste test account passwords to an AI
• Have development reply "can't reproduce" and redo the whole process all over again

🎒 Preparation before you start

• Read 3.2 Skills and Teaching — step 5 of this lesson saves the reproduction workflow as a skill, and this is also a common prerequisite for most scenario pages
• Have a Bot you can assign work to: if you don't have one yet, first go to 1.5 Create Your First Bot
• A staging environment where bugs can be reproduced, plus a fresh test account (not a production account)
• Access to an issue tracking system (Linear, GitHub, etc. — connector or web version both work)
• About 12 minutes, plus one round of human feedback after the first reproduction pack comes out

Follow along with me

Step 1: Write the Bug Reproduction Specialist's responsibilities and boundaries into the Bot's description

Why: The first step of the official six-step method is to write the responsibilities, source systems, output format, and standing boundaries into the Bot's description—the two red lines of this role (staging only, no production data) must also be hard-coded here.

How to do it: Create or open a Bot (sidebar + → Create a Bot), and write the description like this:

You are my Bug reproduction specialist, responsible for turning bug reports into reliable reproduction packages.
Data sources: the issue tracking system and the staging environment.
Output format: a reproduction package—exact steps, expected vs. actual behavior, screenshots, browser and system information;
Include a minimal test case if you can distill one; list anything you cannot verify separately.
Standing boundaries: reproduce only in staging with brand-new test accounts; never use any production customer data;
Issues are read-only; paste the reproduction package into the conversation and wait for my review.

You should see: The description is saved on this Bot's details page. Writing the boundaries here means every future reproduction package follows the same red line.

Step 2: Connect the issue tracking system and the staging environment

Why: The facts in a reproduction package can only come from the systems you connect—the two types of connections the official guide names for this role are: the issue tracking system and the staging environment.

How to do it: Handle the two types of systems separately; for details of the 5-step connection method see 2.5 Connecting apps:

1. Issue tracking system: First search in Connect apps (the official docs' example of a Bot installing an app on your behalf is Linear—saying "Add the Linear app" works too); for systems that don't come up, have the Bot log into the web version using the browser on its cloud computer
2. Staging environment: Your own test environment most likely has no connector—have the Bot open it with the browser on its cloud computer, and store the login credentials in Vault, where the Bot fills them in for you but cannot see the values
3. If a system provides an MCP server, the Bot can also add it by URL
4. If authorization is interrupted, click Reconnect (on the app) or Reopen (on the connection card in the chat) to get back to the login page

You should see: The Bot can open both. Note that Added ≠ logged in—when Needs auth or Disconnected is shown, click Authenticate to re-authorize; after logging into staging once in the Bot's browser, the session persists (the browser is shared, so if one Bot logs in, other Bots can use it too).

Step 3: Test credentials go through a secure handoff, not into the chat

Why: The official guide's special requirement for this role, in its own words, is "Provide test credentials through a secure handoff, not chat"—credentials in the chat log end up traveling wherever the conversation goes.

How to do it: Pick the route based on the login method:

1. Ordinary username and password: Store the test account in Vault—the Bot fills in the login using what's stored in the Vault; the values never pass through the chat and the Bot cannot see them
2. Steps that forms can't handle, like SSO or verification codes: The Bot will hand the computer over to you—on the card labeled Action needed (the Computer card), click Take over, complete the login yourself, then click I'm done to let it continue
3. API-key-type secrets: When the Bot asks, fill them into the secure secret card; the fields are masked and the values are never written into the chat

The official wording: "Never paste passwords or one-time codes into ordinary chat." Also note: fingerprints, Face ID, Windows Hello, and passkeys stored on your laptop or phone are not available on the Bot's cloud computer—if staging goes through your company's SSO, just use Take over.

You should see: The staging login completes and the Bot keeps working; scan the entire chat history and no password or verification code ever appears, from start to finish.

💡 This is the unique value of Grok Bots

Web-based AI either has no credentials to use, or requires pasting passwords into the chat box. The secure handoff of Grok Bots lets credentials go straight from the Vault or your hands into the page, with the Bot never seeing the values—so reproduction packages can be archived and forwarded to developers without worrying that credentials leak along with the package.

Step 4: Run your first AI bug reproduction with the official starter prompt

Why: The official starter prompt deliberately stops at the review point of read-only reproduction—validate the quality of the reproduction package first, then talk about automation.

How to do it: Send the bug report (issue link or full text) to the Bot along with a message, then send this Chinese-localized version of the official Bug reproduction starter prompt:

Read this bug report and reproduce it in staging using a brand-new test account.
Return: exact steps, expected and actual behavior, screenshots, browser and system information,
and if possible, include a minimal test case. Do not use production customer data.

You should see: The Bot executes step by step in the computer preview on the right—click open the preview to watch its clicks, inputs, and navigation. In the end it returns a reproduction package, with screenshots and files attached as cards in the reply that you can click to preview. A qualified reproduction package includes:

• Exact steps (numbered step by step, so others can follow them through)
• Expected and actual behavior (written separately)
• Screenshots (with the relevant state visible)
• Browser and system information
• A minimal test case (the official wording is "if possible"—include it if you can distill one)
• Anything that cannot be verified (per the official requirement to leave an evidence list: state it explicitly, don't make it up)

Step 5: Refine the reproduction package until it's reviewable, then save it as a "bug reproduction" skill

Why: The official six-step method requires refining the results until they're "reviewable without you watching," then saving the workflow as a skill and testing it with a second input.

How to do it: Check the Bot's output item by item—are the steps exact? Are expected and actual separated? Do the screenshots show the relevant state? Is anything unverifiable listed? If it falls short, send it back for a redo; once it's reviewable, solidify the workflow:

Save the complete bug-reproduction workflow you just did as a skill named "bug reproduction".
Include: the source systems (issue tracking + staging), the reproduction steps, the output format (the six items of a reproduction package),
plus the two rules: "never use production customer data" and "credentials only through secure handoff".

The official guidance says a useful skill must spell out six things: when to use it, what inputs and permissions it needs, the order of work, how to verify the results, what it returns, and what requires approval. Once saved, ask for it by name in any conversation—just say "run the bug reproduction skill" to invoke it (@ is for Bots, groups, routines, and apps; it doesn't bring up skills).

You should see: The Bot confirms the skill is saved. The official guidance requires testing it with a second input: run it again with a different bug report and check whether the reproduction package is still solid. The private skill library is shared across all Bots; see 3.2 Skills and teaching for saving details.

Step 6: Define failure handling, then create an issue-event-triggered routine

Why: Step five of the official six-step method: only create the routine after retry and failure scenarios are defined. The routine for this scenario doesn't need a schedule—the official version supports event triggers: Slack messages, GitHub, Linear, Sentry, PagerDuty events, email, webhook.

How to do it: Send the following:

When a new bug report appears in the issue tracking system (GitHub/Linear event),
run the "bug reproduction" skill and paste the reproduction package into this conversation.
If staging is unavailable or login fails, report the failure; do not substitute the production environment.

After sending it, go through the six items of the official confirmation checklist with the Bot: which Bot it belongs to, time and time zone, input source, expected result, approval boundaries, and what to do about missing data—correct it if anything doesn't match.

You should see: the routine appears in this Bot's routines. The new routine doesn't run immediately; it waits for the next trigger—to verify right away, use Test. The routine runs in the cloud, even with your laptop closed; each run consumes usage.

Hold the approval line at this step as well: pasting the reproduction package back to the issue and changing the issue status are external actions—keep them behind approval. An approval card triggered by a routine expires after 30 minutes with no one handling it, so the routine only posts to the conversation; don't configure actions that require approval. For details on approval rules, see 3.1 Approvals and Auto Review.

📋 Magic Prompt

📝 Bug Reproduction Package (official starter prompt structure version)

Expected result: the Bot returns a complete reproduction package—the steps were executed by it firsthand, expected and actual results are separated, evidence is complete, and developers can use it directly.

## Role
You are my Bug reproduction specialist, responsible for turning bug reports into reliable reproduction packages.
## Task
Read this bug report, reproduce it in staging with a brand-new test account, and produce a reproduction package.
## Input Information
### Required Fields
- Bug report: 【issue link or full report text】
- Staging entry: 【staging environment address and login method】
### Optional Fields
- Primary suspicion scope: 【features or recent changes to check first?】
- Known environment differences: 【known differences between staging and production?】
## Output Specification
Reproduction package: exact steps (numbered step by step), expected behavior, actual behavior, screenshots, browser and system information,
minimal test case (if it can be extracted); plus a list of "items that could not be verified."
## Constraints
- ✅ Steps must be ones you executed firsthand, numbered step by step
- ✅ Write expected and actual behavior separately; label facts and inferences separately
- ❌ Do not use any production customer data
- ❌ Do not change issue status or post comments to the issue (posting back to the issue is subject to my approval)

📝 Non-Reproduction Report (a failed reproduction should still produce output)

Expected result: when reproduction fails, you get a structured "could not reproduce" report—what was tried, where it got stuck, and how it differs from the report, instead of a single line saying "couldn't reproduce."

## Role
You are my Bug Reproduction Specialist, and you just failed to reproduce this bug report.
## Task
Produce a non-reproduction report to help determine whether it's an environment difference, missing steps, or an error in the report.
## Input Information
### Required Fields
- Bug report: 【original issue link】
- Steps already attempted: 【recap every step you actually executed and the actual result of each】
### Optional Fields
- Reporter's environment: 【browser/system/account type mentioned in the report?】
## Output Specification
Non-reproduction report: the complete steps actually executed, staging environment information, points of difference from the report's description,
the most likely explanations (labeling which are facts and which are inferences).
## Constraints
- ✅ Label facts and inferences separately
- ✅ Explicitly list the parts that could not be verified
- ❌ Do not fabricate results to claim "successful reproduction"
- ❌ Do not use production customer data

🔧 Reproduction Package Quality Feedback (write the standards into the skill)

Expected result: you point out what this reproduction package is missing, and the Bot writes the requirements into the "bug reproduction" skill, so the next one is noticeably more solid.

## Role
You are my Bug Reproduction Specialist, and you have just delivered a reproduction package.
## Task
Update the reproduction standards of the "bug reproduction" skill based on my feedback.
## Input Information
### Required Fields
- Problems with this package: 【missing screenshots / skipped steps / incomplete environment information / expected and actual not separated…】
- Correct standards: 【what you want every future reproduction package to achieve】
### Optional Fields
- Additional requirements for specific types: 【e.g., payment-related bugs must include timestamp and time zone?】
## Output Specification
First restate the new standards you've summarized; update the skill only after I confirm.
## Constraints
- ✅ Standards written into the skill, taking effect from the next reproduction
- ✅ Only add standards; do not change approval boundaries
- ❌ Do not cut evidence items for convenience

Checkpoint ✅

• Both the issue tracking system and the staging Bot open fine: the connector shows Installed (not Needs auth / Disconnected), and the staging webpage stays logged in
• The first reproduction package is complete: exact steps, expected and actual behavior, screenshots, browser and system information, minimal test case (optional), list of items that could not be verified
• No test account passwords or verification codes found in the chat history—login went through Vault auto-fill or a secure Take over handoff
• The reproduction workflow is saved as a skill; with a different bug report, it can be rerun by name
• No production customer data was touched throughout; the routine only posts the reproduction package, and the action of posting back to the issue stays behind approval

Pitfall Reminders

❌ Pasting test account passwords directly into chat

The official documentation explicitly requires test credentials to go through a secure handoff; the exact words are "Never paste passwords or one-time codes into ordinary chat." The correct approach: store the test account in Vault and let the Bot auto-fill it; when you hit SSO or a verification code, on the Action needed-marked Computer card, click Take over and enter it yourself, then click I'm done when finished.

❌ Letting the Bot "conveniently" reproduce in the production environment

The boundary in the official starter prompt states: "Do not use production customer data." The reproduction package will be archived and forwarded to developers—once production data gets into the package, it travels everywhere with it. Staging plus a brand-new test account is an official requirement; if staging is unavailable, have the Bot report the failure rather than falling back to production.

❌ The Bot says "reproduced," but the steps are copied from the bug report

The value of a reproduction package lies in the steps having been executed by the Bot firsthand. How to check: every step has a screenshot or actual result, expected and actual are written separately, and a list of "items that could not be verified" is attached. Only a reproduction package with every step actually run can be used directly by developers for diagnosis; a "reproduction package" copied from the report is just a bug report in a different format.

❌ Passkeys / fingerprints from your laptop don't work on the Bot's computer

The official troubleshooting page states clearly: Fingerprint, Face ID, Windows Hello, and passkeys saved on your laptop or phone are not available on the Bot's cloud computer. When staging goes through the company SSO, on the Computer card click Take over directly, log in using a method the company allows (password plus verification code, phone push), then click I'm done to hand it back.

Advanced Tips (Optional)

• Auto-run on new bugs: officially supported event trigger sources include GitHub, Linear, Sentry, and PagerDuty—set up an event routine for the "bug reproduction" skill, and as soon as a new bug report comes in, a reproduction package is produced automatically, waiting for your review. See 3.3 Routines.
• Archive the reproduction package as Pages: have the Bot organize the reproduction package into a Pages page (officially positioned as a readable page of this kind of report) and store it in the Bot's Library—looking back at the environment info of a bug at the time is faster than digging through chat history. See 2.2 Files and reviewable results.
• A failure to reproduce is also an output: a structured non-reproducible report directly helps the reporter fill in missing information—environment differences, missing steps, and report errors are listed separately, and posting it back to the issue for approval is all it takes.
• Hand off to performance investigation: if the reproduction package points to a performance issue like "it got slower," hand it to the next lesson B2 Performance investigation—combine it with observability tools to look at traces and locate hotspots with evidence.

Lesson recap

• ✅ A "Bug reproduction specialist" Bot + two types of data sources (issue tracking, staging) that turn bug reports into reliable AI bug reproduction packages
• ✅ Start with the official starter prompt: a fresh test account reproduces step by step in staging—exact steps, expected vs. actual, screenshots, environment info, without touching production customer data
• ✅ Test credentials go through secure handoff: Vault autofill or Take over with manual entry, never entering the chat
• ✅ Save the skill once it's reviewable, define failure handling before creating an incident routine; posting back to the issue always stays behind approval

Key item	Value of this lesson
Bot role	Bug reproduction (bug reproduction specialist)—turning reports into reliable reproduction packages
Connected apps	Issue tracking system (connector or browser login), staging environment (cloud computer browser + Vault)
Starter prompt	Official Bug reproduction starter prompt (localized Chinese version)
Credential handoff	Secure handoff: Vault autofill / Take over (Computer card Action needed → I'm done), never entering the chat
skill	"Bug reproduction" (official six elements, invoked by name, shared across all Bots)
routine	Triggered by a new bug event (optional), posts only the reproduction package, reports failure if staging is unavailable
Approval line	Never touch production data; posting back to the issue / changing status stays behind approval; the routine has no approval actions configured

Next lesson preview

In the next lesson B2 Performance investigation: when the reproduction package points to a performance issue, it's still the same six steps, just swapping the information sources for observability, analytics, and incident tools. You will learn:

• Turning "checkout got slower" into an evidence-backed targeted investigation: look at dashboards and traces, and pinpoint the hotspot with the highest confidence
• Keep facts and assumptions separate, with a short report including screenshots and direct links
• No alert changes, no touching production settings—the investigation is read-only, and changes stay behind approval