Mobile collaboration layers

Mobile co-apping

Mobile co-apping lets an agent observe a visitor use your app in real time. By default, the SDK captures the content of your app’s own views. You can also switch it to capture everything on the screen, including content outside your app. Co-apping then continues while your app is in the background, for example when the visitor switches to another app.

For the configuration properties that govern mobile co-apping, refer to Configuring the mobile co-apping collaboration layer.

If a mobile co-apping action fails, it reports a MobileCoBrowsingErrorType. For the error types and how to handle them, refer to the Runtime errors section of Logging and error handling.

In-app co-apping

In-app co-apping is the default capture mode. The SDK takes a screenshot whenever the content of your app changes and transmits it to the other conversation participants. It captures only your app’s own views, so co-apping pauses when the visitor sends your app to the background.

This capture mode requires no additional permissions.

Out-of-app co-apping

Out-of-app co-apping captures everything on the screen, not just your app’s own views. Co-apping keeps running when the visitor switches to another app. This lets agents guide visitors through device settings, a third-party app, or any other screen. Capture stops when the visitor closes your app or ends the session.

To capture beyond your app, the SDK uses the Android MediaProjection API, which needs the visitor’s consent and a foreground service. For the permissions involved, refer to Capturing beyond your app.

You select the capture mode with the MobileCoBrowsingModule.setCaptureMode method, passing one of the CoBrowsingCaptureMode values:

SCREENSHOT captures your app’s own views. This is the default.
MEDIA_PROJECTION captures everything on the screen.

You can set the capture mode before or during a session. If you switch to MEDIA_PROJECTION while co-apping is active, the visitor is prompted to allow your app to capture the screen. Switching back to SCREENSHOT stops capturing the device screen. Co-apping then only captures your app’s own views again.

When the agent marks an area of the visitor’s screen, the marker isn’t shown outside your app by default. To display it on top of other apps, call setAgentMarkerVisibleOutsideApp with true. The SDK then requests the "Display over other apps" permission when co-apping starts. This setting only takes effect in MEDIA_PROJECTION mode.

Listing 1. Enabling out-of-app co-apping

// The mobile co-apping module you created and registered during initialization.
// See "Code setup and initialization" for how to obtain it.
val coBrowsingModule = MobileCoBrowsingModuleProvider.create()

// Switch capture to everything on the screen instead of just your app's own views.
// Kotlin exposes the Java setCaptureMode(...) method as the `captureMode` property.
// The visitor is prompted to allow screen capture the next time it starts.
coBrowsingModule.captureMode = CoBrowsingCaptureMode.MEDIA_PROJECTION

// Show the agent's marker on top of other apps while the visitor is outside your app.
// This triggers a request for the "Display over other apps" permission when co-apping starts.
coBrowsingModule.isAgentMarkerVisibleOutsideApp = true

Private views and areas

Private views and areas apply to mobile co-apping in both capture modes. In out-of-app co-apping, they only take effect while your app is in the foreground. Once the visitor switches to another app, Unblu can no longer hide content on the screen.

The SDK provides a simple way to add specific views that shouldn’t be shown on the remote side. To do this, the MobileCoBrowsingModule provides the function addPrivateView that takes an ID as its argument. Each view has its own ID.

After a view ID is added, it’s automatically overlaid by an image before the screen is transmitted to the remote side. You can make a view visible to the remote side again by calling the removePrivateView function.

The SDK also provides a way to specify areas which shouldn’t appear on the remote side. The MobileCoBrowsingModule includes the setPrivateArea method that takes a custom ID, values for the x and y axes, and values for the width and height of the private area. Each private area has its own ID that you can call the method with again to update the area’s values. This is especially useful if your app is a hybrid app or is built with a cross-platform framework that doesn’t have views with IDs.

Once an area is added, it’s automatically overlaid by an image before the screen is transmitted to the remote side. To make an area visible to the remote side again, call the function removePrivateArea.

Animations and private views and areas

If you use animations to transition from a view that contains a private view or area, the overlay may not conceal the content during the entire transition.

If you know your app uses animations in views that contain private views or areas, you should take measures to ensure that the content isn’t exposed during the animation. For example, if you know the animation is always horizontal, you might make the overlay larger, so that it covers the area the concealed content traverses during the animation.

Screen sharing publishes everything on the visitor’s screen to the conversation as a screen sharing layer. The shared screen appears in the agent’s screen sharing view, just as it does for screen sharing on the web.

On mobile, screen sharing works in one direction only: a visitor can share their screen with the other conversation participants, but the SDK can’t display a screen that another participant shares.

Screen sharing on mobile uses LiveKit, which you must set up as described in Setting up screen sharing with LiveKit. The device screen is captured through the same mechanism as out-of-app co-apping. Refer to Capturing beyond your app for the permissions involved.

Collaboration layers are typically launched by another participant. You can, however, start and stop screen sharing with the conversation’s startScreenSharing and stopScreenSharing methods. A visitor can start screen sharing from your app’s own UI, without first opening the conversation in the Unblu interface. If the conversation isn’t open, the SDK opens it in the background so screen sharing can start without switching the visitor away from your app.

To track whether screen sharing is active, subscribe to isScreenSharingActive. Alternatively, you can read the current state with isScreenSharingActiveValue.

If starting or stopping screen sharing fails, the ScreenSharingExceptionCallback reports a ScreenSharingErrorType. For the error types and how to handle them, refer to the Runtime errors section of Logging and error handling.

Listing 2. Starting screen sharing and handling errors

// `conversation` is the UnbluConversation you want to share the screen from.
// See "Working with the Unblu Android mobile SDK" for how to obtain it.
// Kotlin passes the two Java callback interfaces as lambdas.
conversation.startScreenSharing(
    {
        // Screen sharing is now active.
        // Reflect it in your app's UI, for example by showing a "stop sharing" button.
        showStopScreenSharingButton()
    },
    { type, details ->
        // Screen sharing couldn't be started.
        when (type) {
            ScreenSharingErrorType.SCREEN_SHARING_NOT_CONFIGURED ->
                // The account or conversation doesn't allow screen sharing.
                showMessage("Screen sharing isn't available in this conversation.")
            ScreenSharingErrorType.UNBLU_CLIENT_NOT_INITIALIZED ->
                // The Unblu client isn't ready yet. Initialize it, then try again.
                showMessage("Please try again in a moment.")
            ScreenSharingErrorType.UNEXPECTED_EXECUTION_ERROR,
            ScreenSharingErrorType.FATAL_ERROR ->
                // Unexpected failure. Log `details` to help with troubleshooting.
                Log.e("Unblu", "Screen sharing failed: $details")
        }
    }
)

Before the SDK can start screen sharing, an administrator must enable mobile screen sharing on the Collaboration Server with the com.unblu.mobiledevice.mobileScreenSharingEnabled configuration property. Otherwise, startScreenSharing fails with SCREEN_SHARING_NOT_CONFIGURED.

For the full set of screen sharing configuration properties, refer to Configuring the screen sharing collaboration layer.

Capturing beyond your app

Out-of-app co-apping and screen sharing both capture everything shown on the screen, including content outside your app such as other apps and system screens. To do this, they use the Android MediaProjection API.

To keep capturing while your app is in the background, the SDK runs a foreground service. The permissions it needs are FOREGROUND_SERVICE and FOREGROUND_SERVICE_MEDIA_PROJECTION. Each capture path declares the foreground service and these permissions, and merges them into your app automatically:

The mobile co-apping module (mobilecobrowsingmodule) provides them for co-apping.
The LiveKit module (livekitmodule) provides them for screen sharing.

You needn’t add anything to your app’s manifest.

Each time capture starts, Android shows a system dialog asking the visitor to allow your app to capture the screen. Capture begins only after the visitor allows it. The dialog is part of Android. Your app can’t change or skip it.

If the visitor denies permission, out-of-app co-apping continues capturing your app’s own views, and screen sharing doesn’t start.

Privacy and masking

When capture extends beyond your app, Unblu can’t control what the device shows. The private views and areas you define only hide content while your app is in the foreground. Once the visitor switches to another app, its content is captured as it appears, and screen sharing captures everything on the screen.

For content that must never be captured, rely on Android’s own protection rather than Unblu’s masking. An app can mark a window with the FLAG_SECURE flag, and Android renders it as black in every screen capture, including co-apping and screen sharing. Banking apps and password managers commonly use this flag.

Document collaboration

Document collaboration lets an agent present a document to the visitor during a conversation. The visitor can view the document and, depending on the conversation’s native document collaboration configuration, annotate and sign it. Document collaboration is provided by the DocumentCoBrowsingModule, which renders documents natively using the Apryse Android SDK.

Registering the module

Create an instance of the module with DocumentCoBrowsingModuleProvider.create and register it on your UnbluClientConfiguration.Builder before you create the client:

Listing 3. Registering the DocumentCoBrowsingModule

val documentCoBrowsingModule = DocumentCoBrowsingModuleProvider.create()
configurationBuilder.registerModule(documentCoBrowsingModule)

Hosting the document viewer

The SDK can display the document in its own activity, or you can host the document viewer in an activity of your own.

By default, the SDK shows the document in its own activity. To allow Android to start that activity, declare it in your app’s AndroidManifest.xml:

Listing 4. AndroidManifest.xml entry for the document viewer activity

<activity
    android:name="com.unblu.sdk.module.documentcobrowsing.internal.OverlayDocumentActivity"
    android:exported="false"
    android:theme="@style/Theme.AppCompat.Light.NoActionBar" />

Alternatively, call useViewerFragmentHost to host the document viewer in an activity of your own. Pass the ID of the fragment container that hosts the viewer and the activity that contains it:

Listing 5. Hosting the document viewer in your own activity

documentCoBrowsingModule.useViewerFragmentHost(R.id.document_container, activity)

When you host the viewer yourself, you needn’t declare the SDK’s activity in your manifest.

Controlling the document collaboration session

The DocumentCoBrowsingModule provides methods to control the document collaboration session:

showDocumentCoBrowsing(callback) presents the document collaboration UI on the device. Use it to show the UI again if it was dismissed while the session is still active.
stopDocumentCoBrowsing(stopLayer, callback) closes the document collaboration UI.
- If stopLayer is true, the call also stops the collaboration layer. This ends the document collaboration session for all participants.
- If stopLayer is false, the collaboration layer remains active and the session continues.
isDocumentCoBrowsingActiveValue() returns whether there’s an active document collaboration session in the conversation.
isDocumentCoBrowsingActive() returns an Observable that emits the current state and any subsequent changes.

The methods that take a DocumentCoBrowsingCallback report the outcome using its onSuccess(boolean) method.