The content below is taken from the original (How the Focused Inbox Replaces Clutter Inside Office 365), to continue reading please visit the site. Remember to respect the Author & Copyright.
The introduction of the “Clutter” mechanism in 2015 inside Office 365 provoked mixed emotions among users. A product of the machine learning work done by Microsoft Research, Clutter attempts to identify messages that are unimportant to a mailbox owner and moves them out of the Inbox into the Clutter folder. The idea is that users then see what’s important in front of them rather than being faced with an overflowing Inbox that’s cluttered up with unimportant notifications, update messages, marketing bulletins, and so on.
Over time, people train Clutter to become more accurate by moving items into the Clutter folder to indicate that they’re unimportant. Conversely, they can mark messages that end up in the Clutter folder as important by moving them back to the Inbox or another folder. Clutter certainly works, but perhaps doesn’t deliver the best end-user experience. Microsoft thinks that the “Focused Inbox” is a better way forward. The roll-out of Focused Inbox to First Release tenants is now in progress within Office 365 and will be supported by all Outlook clients. All Office 365 tenants will receive Focused Inbox in the near future.
Cluttered Confusion for Users
The early days of Clutter were frustrating. Many users reported that it seemed like Clutter “forgot” what it had been taught on a regular basis and had to be continually retrained before the lessons stuck and email was filtered on a consistent basis. After a year or so of using Clutter, I was happy with its operation and confident that the messages that ended up in the Clutter folder were those that I could safely ignore until time allowed me to scan the folder to see if anything important was there.
Time and organizational politics have a habit of changing strategy. Microsoft acquired Acompli and its well-regarded iOS and Android email clients in late 2014. The clients were re-branded as Outlook for iOS and Android and relaunched in January 2015. In part, Acompli built its reputation around the “Focused Inbox,” a mechanism that also aimed to filter messages for users so that the most important were highlighted. The mechanism used to filter messages was simpler than Clutter, but it delivered a great user experience that worked well, especially on mobile devices.
Applying Intelligence to Email
The result is that Microsoft had two different approaches for intelligent email filtering. The client-side approach implemented in the Outlook apps delivers a good user experience and is very popular with users. The server-side approach implemented in Clutter is potentially more powerful and flexible in terms of machine learning and extensibility, but has not achieved the same degree of acceptance with users. Microsoft’s solution is to combine the best of both into a new unified Focused Inbox experience that is available for all Outlook clients — desktop, mobile, Mac, and browser.
One thing we have learned from previous changes made to Outlook is that when change is introduced, some potential for disruption exists. The best example of this is the introduction of the ribbon in Office 2010. Given that the Focused Inbox affects the way that Outlook works in a pretty fundamental way, it’s clear that some understanding is required by both administrators and users if disruption is to be avoided.
Marking Messages as Important
The basic idea embodied in the Focused Inbox is that Inbox items are separated into two views. Items deemed to be important are shown in the Focused view, while all other items are shown in the Other view, where that they can be left until the user has some time to process them. Behind the scenes, Exchange Online uses a MAPI property (tag 0x1213003) to indicate the view to which an item belongs. That property, which is visible with the MFCMAPI utility, is stamped on items when they are delivered to the Inbox and processed by algorithms that assess the characteristics of messages to make the decision as to which view an item belongs.
Figure 1 illustrates what you can expect to see through MFCMAPI. In this case, an item is selected from the Inbox folder and the value of tag 0x1213003 is 0 (zero), indicating that the item belongs to the Focused Inbox view. If the tag value is 1 (one), the item belongs to the Other view. The code to stamp messages was introduced to Exchange Online servers some time ago to ensure that the views are immediately available to users who choose to use the Focused Inbox. Because items that end up in the old Clutter folder have gone through much the same decision process as applied to create the Focused Inbox views, it’s logical that these items are stamped with a value of 1 in tag 0x1213003.
Figure 1: Using MFCMAPI to view the Focused Inbox tag (image credit: Tony Redmond)
Rules Rule
Inbox rules have been supported by Exchange for a very long time. Some users are devoted to using rules to organize their mailbox and have built their own set of rules to process inbound email. Because they are created by a user and are assumed to be an absolute directive as to how certain messages are processed, rules have priority and are executed against new email by Exchange Online first. New items are then subject to Focused Inbox classification.
Acquiring Knowledge of How Users Process Email
Training for the Focused Inbox is done by acquiring information about the characteristics of messages deemed by the user to be important. Like Clutter, more than 40 different characteristics of a message are measured to determine its importance, including the sender, the recipients, the subject, priority, and so on. Over time, the set of messages deemed by the user to be important creates a more reliable training model against which new messages can be assessed.
Clutter takes time to become effective for new mailboxes, as it needs to assess up to one thousand inbound messages to create a training model before it can refile messages. The same kind of training occurs for the Focused Inbox, but to make the feature more useful for new mailboxes, a basic set of rules is applied to messages from the start. For instance, anything that looks like a notification message is directed to the Other view, whereas mail from someone’s boss (using a lookup against Azure Active Directory) is considered important and remains in the Focused Inbox.
Outlook clients support other mail systems such as Gmail and Outlook.com in addition to Exchange Online and Exchange on-premises servers. The same classification model is used for all servers and is implemented within Office 365 using the new architecture that Microsoft has deployed to support background processing for mobile clients. Microsoft plans to make the Focused Inbox available to users of on-premises Exchange and non-Microsoft email servers, but the company hasn’t yet fully worked out when this will be possible.
If you use Clutter today, the transition to the Focused Inbox is easy. All of the work you did in the past to train Clutter is carried over to the Focused Inbox. Items that are in the Clutter folder remain there until you remove them. After you begin using the Focused Inbox, the Clutter folder loses its status as a system folder, which means that you can delete all items from the folder and then remove it. In fact, if the Clutter folder is empty when a mailbox switches to the Focused Inbox, the folder is removed automatically.
Client Updates for the Focused Inbox
Obviously you can’t use the Focused Inbox without a client that supports the feature. Microsoft will provide updates for Outlook 2016 (the click to run version first, followed by an update for the MSI version), Outlook 2016 for Mac, Outlook Web App, Outlook Mobile, and Outlook for Windows 10 Mobile (and its UMP counterpart running on Windows 10 workstations). The feature is not going to be backported to Outlook 2013.
Because the item property governing the Inbox view to which an item belongs can be read by the majority of the client protocols supported by Exchange (MAPI, EWS, and Microsoft Graph, but not IMAP4 or POP3), clients can construct their own versions of the views as deemed appropriate for the hardware form factor and client user interface in use. The property is reset if an item is moved from one view to the other. Because this action can be taken on any client, it’s possible to use any client based on a supported protocol to train the algorithms that make the decision about how to categorize new email.
Exchange ActiveSync (EAS) is a notable exception, as this protocol does not support the concept of folder views. EAS is built around the storage paradigm of items held in folders, and therefore retrieves and displays information on that basis. EAS is the lowest common denominator access protocol for Exchange mailboxes. Even if Microsoft released an updated version of the protocol to support folder views, no guarantee exists that any of the mobile email client vendors who license EAS (such as Apple and Samsung) would ever implement views.
How Users Switch to Focused Inbox
The Clutter feature remains enabled for mailboxes even when Focused Inbox is enabled for the tenant. This sounds a little strange, but there is good logic as to why this is the case. Two important facts apply. First, a client update is necessary before users see the Focused Inbox views in action. Second, Focused Inbox and Clutter leverage much the same learning model. Inbound messages are categorized for both Clutter and Focused Inbox.
The right time will eventually come to move to the Focused Inbox. To make things easier for users, some code in the client-server handshake for the updated OWA and Outlook clients detect when Focused Inbox is available and can perform the switch over for the user. Figure 2 shows the option presented by OWA.
Figure 2: How OWA offers the option to switch to Focused Inbox (image credit: Tony Redmond)
Switching is painless and takes just a few seconds to become active. Clutter is disabled and the user sees the Focused and Other views at the top of OWA’s message list. Changing from one view to another is a simple matter of clicking the view you want to see. One nice feature is that notifications appear at the top of the message list when new items arrive and are placed into the other view (Figure 3). As you can see, 3 new items are waiting for attention in the Other view. If you’re in the Other view when new important messages arrive, you’ll be told that new messages await in the Focused view.
Figure 3: The arrival of new messages in the other view are flagged at the top of a message list (image credit: Tony Redmond)
If a message arrives into the Focused view that should not be there, the user can flag the message as unimportant by using the “Move to Other inbox” option in the context-sensitive menu exposed by a right click (as shown in Figure 4). You can either move the selected conversation or all messages from the sender (Always move).
Figure 4: Marking an unimportant message to be moved to the Other view (image credit: Tony Redmond)
It’s also possible to disable Clutter for a mailbox explicitly by running the Set-Clutter cmdlet. However, if you do this, the mailbox owner will see messages that were previously redirected to the Clutter folder remain in the Inbox and they’ll have to process those messages until their client is upgraded to a version that supports the Focused Inbox. It’s therefore best not to disable Clutter until the upgraded clients are available. But if you want to disable Clutter, here’s an example of the required command:
[PS] C:\> Set-Clutter -Identity TRedmond -Enable $False
In the event that someone prefers to continue using Clutter, you can disable the Focused Inbox for their mailbox and make sure that Clutter is enabled to revert back to the older feature. You might want to do this to ensure that unimportant email is kept out of the user’s Inbox while they wait for a client update. Clutter will continue working until Microsoft finally deprecates Clutter (expected in about a year or so), at which time Clutter will be removed from Office 365.
Management of the Focused Inbox
Tenants can manage the Focused Inbox feature at a tenant or at a mailbox level. You can enable or disable the feature for the tenant by updating the Exchange Online organization configuration. The default state is that the Focused Inbox is enabled, which can be checked by running this command:
[PS] C:\> Get-OrganizationConfig | Format-Table FocusedInbox*
FocusedInboxOn FocusedInboxOnLastUpdateTime
-------------- ----------------------------
1/1/0001 12:00:00 AM
The FocusedInboxOn setting indicates the overall state of the feature. As you’d expect, this is True when enabled or False when disabled. If the value is not set, the mailbox-specific settings take precedent (see below). FocusedInboxOnLastUpdateTime is a timestamp recording when an administrator last updated the FocusedInboxOn setting. It’s quite normal to see a blank value for FocusedInboxOn and a rather strange date value for the last update time stamp, as shown in the example. This is because Microsoft doesn’t explicitly enable the Focused Inbox for any tenant and a blank value means that it’s enabled by default. In fact, the tenant configuration setting really exists to allow a tenant to block the Focused Inbox if required. To turn Focused Inbox off for a tenant, you use the command:
[PS] C:\> Set-OrganizationConfig -FocusedInboxOn $False
And to turn it back on:
[PS] C:\> Set-OrganizationConfig -FocusedInboxOn $True
After switching the Focused Inbox back on, you’ll see that the organization configuration has changed:
[PS] C:\> Get-OrganizationConfig | Format-Table FocusedInbox*
FocusedInboxOn FocusedInboxOnLastUpdateTime
-------------- ----------------------------
True 10/11/2016 1:22:10 PM
Control over the Focused Inbox can be exerted over individual mailboxes using the Set-FocusedInbox cmdlet. For example, here is how to disable the feature for a mailbox:
[PS] C:\> Set-FocusedInbox -Identity TRedmond -FocusedInboxOn $False
If the property is not set for a mailbox, the tenant configuration applies. If an explicit value to control the Focused Inbox is not set at either the mailbox or organizational level, the Focused Inbox is deemed to be enabled.
Bypassing the Focused Inbox
Unlike Clutter, the Focused Inbox does not send notifications to users telling them about messages that were recently categorized as unimportant and moved to the Clutter folder. The notifications are unnecessary because the two views are always available in the Inbox.
Like Clutter, an administrator can create conditions to mark messages that should always be deemed to be important, no matter what the user thinks. For instance, messages sent by the CEO or the HR department are always important. Both Clutter and the Focused Inbox use transport rules to mark important messages with a header that can be used to recognize their status. Here’s an example that stamps messages with “CEO Communication” with the header used by Focused Inbox.
New-TransportRule -Name “Focused Inbox Bypass” -SubjectContainsWords "CEO Communication" -SetHeaderName "X-MS-Exchange-Organization-BypassFocusedInbox" -SetHeaderValue "true"
To check that the rule works, create a message with the bypass keywords in the subject and send it to a user. Use a tool such as MessageHeaderAnalyzer (shown in Figure 5) to check the received copy in the user’s mailbox and look for the X-MS-Exchange-Organization-BypassFocusedInbox: header, which should contain a value “true”. See this article for more information on viewing message headers.
Figure 5: Checking a message header to validate that the bypass rule is working
Clutter uses a similar process to enable important messages to pass untouched. To ensure backward compatibility, any message stamped with the old X-MS-Exchange-Organization-BypassClutter header will go into the Focused Inbox.
Some Issues to Think About
Folders are less important to mobile clients than they are in the desktop version of Outlook. It therefore makes eminent sense to use views to organize messages on those platforms. Swapping out a folder, as used by Clutter, for a view seems quite a small step, but the change has some consequences that might not be expected. For example:
- Until updates for mobile clients are available, ActiveSync-based clients will download all messages from the Inbox, including those deemed to be unimportant, which might lead to additional work to process those messages as well as extra synchronization activity (only really an issue across slow networks).
- The Clutter folder is a system folder and a retention tag can be applied to the folder that is different to the retention tag applied to the Inbox. For instance, you might delete items from the Inbox after 120 days but use a 30-day retention period for Clutter. That’s not possible now because retention tags can only be applied to folders, not views within folders.
- Outlook rules created by mailbox owners don’t take the Focused Inbox views into account. You can’t, for instance, create a rule to move specific messages into the Focused or Other view.
- The PowerShell cmdlets that operate against Exchange Online mailboxes are built to expect folders rather than views. This shouldn’t cause too many problems, but it is something to remember. For example, if you run Get-MailboxFolderStatistics against the Inbox, the data returned is for the Inbox as a whole and is not broken down by view.
- Searches (eDiscovery, those performed with the Search-Mailbox cmdlet, or Office 365 content searches) do not indicate whether found items belong to a specific view. If you want to find out this level of detail about a found item, you need to examine its properties with the MFCMAPI utility.
In Summary
Any action to rationalize user experiences and capabilities across all versions of Outlook is goodness. The Focused Inbox has already proven itself in the Outlook for iOS and Android apps and there’s no reason to suspect that it would be any less effective across the complete Outlook family. Some might prefer the way that Clutter operates but the evidence from user forums is that most do not. Its passing is a blip on the path to progress. Microsoft hopes that the Focused Inbox generates less heartburn and confusion for users. The early signs are positive.
Follow Tony on Twitter @12Knocksinna.
Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros,” the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.
The post How the Focused Inbox Replaces Clutter Inside Office 365 appeared first on Petri.
