Mod Assist

Why should you integrate Rocket.Chat with Mod Assist app

Content moderation serves as a crucial safeguard against harmful and offensive messages, ensuring the well-being of platform users. Businesses have a moral responsibility to shield their users from inappropriate content, maintain their brand's reputation as a secure environment, and adhere to relevant regulations that mandate the removal of offensive and illegal material.

However, content moderation poses challenges, particularly due to the sheer volume of messages continuously exchanged on collaboration platforms. Additionally, the potential risks of exposing human moderators to harmful content make manual moderation increasingly less desirable. Here are a few limitations associated with manual content moderation:

• Volume: The rapid proliferation of collaboration platforms and the ever-increasing number of messages exchanged make it difficult for businesses to keep pace with the sheer volume of content requiring moderation.
• Consistency: Ensuring consistent moderation practices across an organization can be a daunting task, especially when multiple moderators or cultures are involved.
• Accuracy: Moderators must possess the ability to accurately identify inappropriate or harmful content and make informed decisions regarding its approval or rejection.
• Speed: Content moderation is a complex task, especially when it comes to dealing with gray areas and ambiguous situations. One key aspect is the speed at which content needs to be moderated, as quick action can help minimize the impact of inappropriate or harmful content.

Typically, content moderation involves a two-step process. First, users report content that they find offensive or in violation of workplace guidelines. Then, a moderation team reviews and assesses these reported messages. The downside of relying solely on user reporting is the risk of harmful content going unnoticed, potentially damaging your platform's reputation and eroding user trust.

This is where automated content moderation comes into play. The Mod Assist App automatically detects and reports inappropriate content based on predefined moderation rules within a workspace.

Note: For privacy reasons, the app is designed not to operate in private rooms by default and consequently, it won't report messages from such rooms.

However, if you wish to enable moderation for these private rooms as well, you will have to individually admit the App into these rooms, as a user. Remember to add these private rooms under the App's settings, in the section where you specify the rooms for the app to moderate. This will serve as two-factor authentication for accessing private rooms.

Benefits of using the Mod Assist app

This app offers numerous benefits, including speed, protection, and efficiency.

• Improved efficiency: By automatically flagging questionable content, the app streamlines the moderation cycle, allowing for a faster and more efficient review. This reduces the workload for human moderators. As a result, they can focus on more complex tasks, such as reviewing flagged content or investigating reports of abuse.
• Increased speed: Considering the rapid proliferation of messages exchanged, relying solely on manual moderation would be impractical, as it would require a large team of human moderators and a tightly packed work schedule. The key advantage of this app is that it saves your moderators valuable time by allowing them to focus on reviewing only the reported messages rather than having to screen every single message and flag the inappropriate ones.
• Better well-being: The app plays a pivotal role in safeguarding the safety and mental well-being of users and human moderators by expeditiously detecting various forms of harmful and offensive content pertaining to violence, sexually explicit material, and potentially illegal activities.

The specific types of content that can be moderated automatically vary depending on the individual platform's needs and available resources. The following categories of inappropriate content can be detected and reported using the app:

• Toxicity: any message that is harsh, disrespectful, negative, malicious, harmful, or hateful is toxic in nature. The app can identify and flag content that contains bigotry and messages that constitute bullying or harassment.
• Profanity: any message where the language is bad, abusive, foul, vulgar, or obscene is deemed profanity. The app is trained to identify and flag content that contains profanity or explicit language.
• Financial Risk: The app can identify and flag scam messages sent by fraudsters and imposters to trick you into divulging sensitive personal information or financial details that can lead to identity theft, financial loss, or unauthorized access.
• Flirtation: messages that are indecent, intimate, unwanted, and inappropriate can be deemed flirtatious content types. 

The Mod Assist app will only be available to workspaces running version 6.3.0 and higher. The Mod Assist app is available for all premium plans. Please refer to our technical documentation to get started with the app.