Skip to main content

Customer Conversation Analysis (Intercom)

Executive Summary​

Analysis of Intercom conversations from January-May 2025 reveals critical customer pain points that correlate strongly with JIRA support tickets, providing a complete picture of customer experience challenges.

Real Customer Issues (From Intercom Data)​

Top Conversation Themes​

1. File & Document Management Issues (35%)​

Common complaints:

  • "we're not able to delete any files from the shared integrations. why?"
  • "How do you delete a file that someone else uploaded?"
  • "what is the page limit I can upload via direct injection. I keep getting an error I am trying to upload a 124 page rfp."
  • "I am having trouble attaching documents that provide context for my prompts"
  • "syncing issue on my end, some documents have not been synced since yesterday"

JIRA Correlation: Maps directly to 25% of SUP tickets related to file embedding/sync

2. AI Model & Performance Issues (30%)​

Direct quotes from customers:

  • "settings no longer shows LLMs to pick from - am i doing somethign wrong?"
  • "My AI has been struggling with a pretty simple task the last 30 minutes"
  • "Hi I am having trouble wording a prompt to create a graphic"
  • "getting 403 error on webscrape. any tips?"
  • "I'm trying to compare two versions of an SOW... I keep getting the red message of death"

JIRA Correlation: Aligns with 35% of SUP tickets about AI model errors

3. UI/UX Frustrations (20%)​

Customer feedback:

  • "I don't love the new way to add tags... This is another step that I will likely now skip"
  • "When I'm in chat and I have my Files open... I can't scroll down"
  • "there is no way to stop [generation]... There is not a 'stop' button"
  • "when right clicking on a session, adding it to a project... is NOT an option"

JIRA Correlation: Matches 20% of SUP tickets on UI issues

4. Integration & Sync Problems (15%)​

Real examples:

  • "Hello, having trouble deleting files in our Shared Integration"
  • SharePoint sync failures
  • Document upload spinning indefinitely
  • Workflow errors with API key issues

JIRA Correlation: Corresponds to 15% of integration-related tickets

Customer Sentiment Analysis​

Negative Sentiment Indicators​

  1. Frustration Language:

    • "red message of death" (used multiple times)
    • "struggling with a pretty simple task"
    • "This is another step that I will likely now skip"

  2. Feature Abandonment Risk:

    • Tag feature: Users explicitly stating they'll skip due to complexity
    • File management: Users unable to complete basic tasks

  3. Productivity Impact:

    • "inhibiting me from completing some tasks"
    • "30 minutes or so with multiple refreshes"

Positive Indicators (Limited)​

  • Users attempting advanced features (workflows, comparisons)
  • Engagement with training/certification programs
  • Active problem-solving attempts before contacting support

Intercom-JIRA Correlation Analysis​

Pattern Matching​

Intercom TopicFrequencyJIRA CategoryTicket VolumeCorrelation
File deletion/managementHighFile sync issues25%Strong
AI errors/"red message"HighModel errors35%Strong
UI navigationMediumUI/UX problems20%Direct
Upload limitsMediumPerformance5%Moderate
Feature confusionHighDocumentation10%Indirect

Customer Journey Mapping​

Key Insights for CTO​

1. Critical UX Failures​

  • Basic operations (delete, upload, scroll) are failing
  • Users can't differentiate between UI bugs and intended behavior
  • Feature complexity driving abandonment

2. Communication Gaps​

  • Error messages unhelpful ("red message of death")
  • No clear status indicators for long operations
  • Missing feedback loops for user actions

3. Permission Model Confusion​

  • File ownership model not understood
  • Shared workspace permissions unclear
  • No self-service permission management

4. Performance Perception​

  • Users experiencing "30+ minute" delays
  • No visibility into processing status
  • Timeout errors without recovery options

Actionable Recommendations​

Immediate (Week 1)​

  1. Error Message Overhaul

    • Replace generic errors with actionable messages
    • Add error codes for support reference
    • Include self-help links

  2. File Management Fix

    • Implement "Request Deletion" for non-owned files
    • Add bulk operations
    • Clear ownership indicators

  3. Stop/Cancel Buttons

    • Add generation stop with save option
    • Implement for all long-running operations
    • Show progress indicators

Short-term (Month 1)​

  1. Simplify Core Workflows

    • Redesign tag addition (single-click)
    • Streamline file upload process
    • Fix scroll issues in sidebar

  2. Status Communication

    • Real-time sync status dashboard
    • Processing queue visibility
    • Estimated completion times

  3. Permission Clarity

    • Visual permission indicators
    • Self-service permission requests
    • Admin delegation options

Long-term (Quarter)​

  1. AI Reliability

    • Implement fallback models
    • Better error recovery
    • Offline mode for critical features

  2. Performance Optimization

    • Async processing for large files
    • Progressive loading
    • Client-side caching

  3. User Education

    • Interactive tutorials
    • Context-sensitive help
    • Feature discovery prompts

Success Metrics​

Leading Indicators​

  • Intercom conversation volume (target: -50%)
  • Error message frequency (target: -70%)
  • Feature adoption rates (target: +30%)

Lagging Indicators​

  • Support ticket creation (target: -40%)
  • Customer satisfaction (target: 4.5/5)
  • Feature usage retention (target: 80% at 30 days)

Risk Assessment​

High Risk Customers (Based on Complaint Frequency)​

  1. Government contractors with strict deadlines
  2. Large document processing users
  3. Team collaboration heavy users
  4. New customers in onboarding

Churn Indicators Detected​

  • Multiple error encounters per session
  • Explicit feature abandonment statements
  • Productivity blocker complaints
  • Comparison to competitor features

Strategic Alignment​

Product Strategy Impact​

  • Need to prioritize stability over new features
  • UX debt is becoming business-critical
  • Enterprise features blocked by basic issues

Engineering Priority Shift​

  • Move from feature development to stability
  • Implement comprehensive error handling
  • Focus on user journey completion

Customer Success Requirements​

  • Proactive monitoring for at-risk accounts
  • Enhanced training on workarounds
  • Regular check-ins for power users

Data Sources:

  • Intercom API conversations (Jan-May 2025)
  • JIRA SUP project correlation
  • Direct customer quotes and feedback