Nextcloud Hub 4 pioneers ethical AI integration for a more productive and collaborative future

Post date

March 21, 2023

That is not all

But there is much, much more. We introduce beta apps for Talk on the desktop and Virtual Files on Mac OS. We introduce recording and breakout rooms in Talk, file attachments in Calendar, advanced versioning in Files, the new Tables app, we bring the Notes app into Nextcloud, make Deck more collaborative – the list is long. Please, read on, share your thoughts on this release with us in the comments or on social media and tell friends, colleagues and family that it is possible to have a great collaboration experience WITHOUT giving your data to Big Tech!

Intelligent features in Nextcloud Hub 4: Introducing the ‘Smart Picker’

The best way to introduce our new, intelligent features is through the Smart Picker – a cross-application feature enabling users to use AI (and much more) in their daily work flow.

For the first time, users have the opportunity to work with AI in a personal and collaborative way.

To activate the Smart Picker, just type a slash (/) in the chat bar and you’re instantly given a wide selection of options.

*Simply type a slash (/) to activate the Smart Picker in Nextcloud Talk*

Note: In order to activate, you must upgrade to Nextcloud Hub 4 (Version 26) and install the integration apps.

With the Smart Picker, you can talk to, create with and work faster thanks to AI.

Talk to AI: Speech-to-text

Sometimes, it’s easier to say something out loud. With Nextcloud’s speech-to-text feature utilizing Whisper via Replicate, you can formulate text through your voice. Use your spoken words to send a message in a Nextcloud Talk chat, in an email and through many more applications.

Example of using AI Speech-to-text (by Whisper via Replicate) in Nextcloud Mail

Create with AI: Image generation

Your cloud is full of photos, but sometimes the image you need isn’t there. Maybe it is somewhere on the web. Maybe you only have it in your head? With AI image generation, you can type in keywords to create the image you’re looking for. Insert an AI generated image into a Talk chat, Notes and much more.

Example of using AI image generation (by OpenAI’s Dall-E 2 and StableDiffusion) in Nextcloud Talk

Nextcloud gives you two options to generate images: the online Dall-E 2 and an on-premises StableDiffusion based generator.

Work with AI: ChatGPT text generation

Recently, all the rage has been centered around the novel ChatGPT, so we’ve decided to integrate its relative right into Nextcloud. ChatGPT, which uses GPT-3 technology by OpenAI, is known for crafting intelligent, professional text and our version is now readily available to use in Nextcloud Talk, Office, Mail and more.

Use the Smart Picker to generate fresh text by giving the tool instructions on what you want to know. For instance, in Nextcloud Talk, ask a question that will help you construct a message to the team you are managing.

The GPT-3 text generation delivers 1 result by default, however the advanced settings allows for as many results as you want and you can even choose the model of data to use for optimal results.

Example of using ChatGPT text generation (by OpenAI) in Nextcloud Office

Smarter collaboration with the Smart Picker

Now that we’ve introduced the AI features and how they work, let’s talk more about the Smart Picker. The Smart Picker was created to enhance your workflow and to better collaborate. It is conveniently integrated across all your Nextcloud applications resulting in seamless collaboration.

The Smart Picker is available in Nextcloud Talk, Deck, Mail, Notes, Text, Collectives, Office and other applications.

In addition to the AI speech-to-text, image generation and ChatGPT text generation, you can use the Smart Picker across all applications to:

Share tasks from a Deck board, card or comment
Share Nextcloud Tables
Share a map location (by OpenStreetMap)

Example of using map location (by OpenStreetMap) in Nextcloud Text

Share a knowledge database items from Collectives pages
Share videos from PeerTube
Create Text templates or pre-defined text snippets

Example of using a Text template in Nextcloud Mail

Share GIFs (by Giphy)
Share Mastadon people, toots and hashtags
Share Movies, series and people (by The Movie Database)

Example of Movies, series and people (by The Movie Database) AI feature

Share Files
Share any link
Share GitHub issues, pull requests and comments
Share GitLab repositories, issues and mergers
Translate with Nextcloud Translate (via DeepL or a local, on-prem translation AI)

Example of the Translation in Nextcloud AI feature

The Smart Picker was designed in a way to be easily expandable, so we look forward to many more apps on the store that can help you insert smart information into your documents, mail and more!

Try the Smart Picker now!

Note: In order to activate, you must upgrade to Nextcloud Hub 4 (Version 26) and install the integration apps.

Ethical AI Rating

We thought long and hard about the ethical issues with AI. As you might remember, our previous releases did all AI related work on the server. We did our very best to find ways to classify images, recognize faces and recommend you files without leaking any data to a third party server.

But AI is moving fast, and we do want to give you the option to take advantage of innovations from third party companies like Deepl, OpenAI and others.

We can’t fix all problems in the world, but we can give you a choice, and provide transparency. So, to address the obvious privacy and control challenges that come with artificial intelligence, the AI-powered features are transparently classified with Nextcloud’s Ethical AI Rating. The Ethical AI Rating is based on the availability of code, model and training data so you can make informed choices. We share all about the conditions, classifications and methodology of the Ethical AI Rating in our blog.

Learn about Ethical AI

Nextcloud Tables: Alternative to Microsoft SharePoint

Use the Smart Picker for Tables in Nextcloud Text

Example using Tables: Vacation Requests

Creating a table

List of your tables

Nextcloud Tables is an on-premises, open source SharePoint alternative that lets customers track data that is relevant to their organization, such as expenses, projects or sales leads.

It enables customers to build, manage and work with data structures, putting together easy ‘apps’ and connecting them to various other components in Nextcloud for automation. The flexibility of Tables allows users to tailor the table to their specific needs.

Get Tables now

“The initiative by Nextcloud to offer a digitally sovereign, open-source alternative to Microsoft SharePoint is commendable. That is why we are working with Nextcloud to optimize Nextcloud Tables.”
Ralf Sutorius, Senior IT Architect, Stadt Köln (City of Cologne)

Notes becomes core

Our community Notes app makes a big step forward today: it becomes Nextcloud Notes, and will be a part of Nextcloud core itself going forward. With this change come many improvements, plus the good news that the mobile apps, both for Android and iOS, are now free!

We use notes for a large variety of things – from shopping lists and planning a vacation to jotting down thoughts during a meeting. Much note taking happens on our phones, which often means the notes themselves are hosted by Google, Apple and other big tech companies. That is why we thought it was important to bring Notes into Nextcloud itself!

Besides making the mobile apps free, the Notes app itself now uses Nextcloud Text for the notes themselves. This brings many of the Nextcloud Text features to Notes, including… our Smart Picker! So all the wonderful features you’ve seen above can be accessed in Text – you can insert videos, Tables, documentation, locations and of course ask ChatGPT to help you create some ideas.

Nextcloud Hub 4 Product Improvements

Files, Talk, Groupware and Office are Nextcloud’s key products and each have many exciting improvements and updates!

What’s more, all products are integrated with the new Smart Picker with AI collaboration that will transform the way you work.

Nextcloud Files

Advanced Versioning

In Nextcloud Files, we kept older versions so you never lose the original text or one in between. With Nextcloud Hub 4, we’re taking it to the next level giving users more control over the smart versioning mechanism in Nextcloud. It is now possible to name a version, so you can easily go back to the original copy or one created by a colleague at a certain date and time.

Name the version

Save version name

Files settings

Nextcloud will keep versions using a time based algorithm. This keeps less precise versions over time, for example, keeping 1 version per minute, 1 per hour, 1 per day and so on. The system also acts on available file storage, cleaning up older versions when storage gets low. But named versions are always kept!

💡 AI component: Recommended files are shown at the top of your files list and you can share them based on old sharing behaviors.

Related resources of Files & Deck in Talk

Desktop client & mobile apps

The Files desktop client has a major release on the way, however a preview version is out now of virtual files on MacOS. As for the upcoming version, a default browser selection is introduced, as well as the ability to create accounts from the command line. That is of course very useful in large organizations.

As for the Android files app, document scanning with multi-page support to PDF and PNG is now available.

The iOS Files app delivers improvements to document scanning, image editing before uploading, and office document editing improvements.

E2EE improvements

End-to-end encryption is designed to keep a sub-set of super confidential data safe, even if a server breach occurs. Imagine one of the hundreds of universities that use Nextcloud. The collaboration between students should be overseen by teachers and system administrators. But the HR department, dealing with very sensitive employee information, should be able to work on documents without even system administrators being able to view them. This is what End-to-End encryption makes possible. We made two improvements in this release.

First, we introduce sharing. Before, after picking a folder for end-to-end encryption, users could work in this folder and access it from all their clients, mobile and desktop, but not share it. It is now possible for the HR team to share their folders with confidential dossiers with each other!

You may already use File Drop – the Nextcloud feature that allows your external personnel, clients and partners to easily upload files to a secure cloud folder through a hyperlink.

With the new Nextcloud Hub 4 release, File Drop now includes the option to upload files to End-to-End Encrypted folders for increased security in your organization.

Receive confidential files in a secure, trustworthy way without worrying about breaches. It’s simple, secure and a no fuss file exchange.

Going back to the HR department – the team can now send job applicants a link where they can securely upload their resume’s directly in an end-to-end encrypted folder, shared in the HR team.

Nextcloud Talk

This release of Talk introduces a series of awesome features. But before we show those to you, we have to tell you we also deliver a much requested feature: a desktop client for Talk! It is not finished yet, but we would love to get your feedback.

You can download the preview here.

Jump start conversation in breakout rooms

Breakout rooms are excellent for diversifying large meetings and virtual learning environments.

In this release, Nextcloud provides full functionality to set up a breakout room, manage it, send messages and much more.

Configure breakout rooms

Manage breakout rooms

Send message to a breakout room

Send message to all breakout rooms

Record and track every call

For everyday work calls, there’s now a recording option with a time watcher. Easily keep track of time in a call with the time watcher and never be out of the loop with session recording. After the call, simply send the recording in a chat and it will pop-up for all participants to see and utilize.

Start a recording

Recording started

Send recording in a chat

Nextcloud Talk reduced loading times, server load and notification delays by up to 99%.

More in Talk

🎮 Several administrator features to control permissions

🔊 Conversation ownership transfer

📱 A dedicated iPad interface

💡 AI component: Use the Smart Picker in Nextcloud Talk for Nextcloud Dictate (speech-to-text), Nextcloud Translate (with DeepL or on-prem), Dall-E (image generation), ChatGPT technology and much more!

Smart Picker view in Talk

Smart Picker view in Talk featuring the AI image generator with Dall-E & StableDiffuse technology

AI speech-to-text

AI image generator with Dall-E & StableDiffuse technology

Image generated in Talk example

Nextcloud Groupware

Shared mailboxes and S/MIME support

We also introduce shared mailboxes so your enterprise or organization can be more efficient and better organize its teams and workflow.

To improve suitability for large enterprise and government organizations, the Mail App in Nextcloud Hub 4 introduces S/MIME (Secure/Multipurpose Internet Mail Extensions) for digitally signing and encrypting messages.

S/MIME is a security protocol that ensures secure email communication so your information is not compromised.

💡 AI component: Use the Smart Picker in Mail for Nextcloud Dictate (speech-to-text), Nextcloud Translate (with DeepL or on-prem), Image Generation (Dall-E), ChatGPT technology and much more!

Smart Picker Mail highlights

Text templates – excellent for recurring sales or support messages

Email text templates

GIF picker (by Giphy) – to lighten the mood in an email chain
Movies, series and persons (by The Movie Database) – Be part of the movie conversation!

Example of using ChatGPT text generation (by OpenAI) in Nextcloud Mail

Other improvements in Groupware

Support for attachments when making an appointment in Calendar
XOAUTH2 support in Mail to improve interoperability with Outlook and Gmail

💡 AI component: The Smart Inbox in Mail is getting improvements, training the local neural network also on mail subjects to be able to better recommend you important emails.

Attachment support when making an appointment in Calendar

Nextcloud Office

New improvements with Text and Collectives

Nextcloud Office has improved integration and performance, document compatibility and a reworked UI.

Real-time cursor tracking

While editing a document, you can now see others who are working on the document too in real-time. As they write and move about the document, you’ll see their name highlighted on their cursor. Watch the additions and corrections others make to better help you work on a shared document.

Collectives

In Nextcloud Office, the knowledge base application Collectives improves the creation of new collectives and moving pages between them. It also links collectives to Files, allowing users to find the files that make up a collective and the collective associated to files more easily.

💡 AI component: Use the Smart Picker in Nextcloud Office for Nextcloud Dictate (speech-to-text), Nextcloud Translate (with DeepL or on-prem), Dall-E (image generation), ChatGPT technology and much more!

Improved text & Smart Picker in Deck

The project management app Nextcloud Deck now also uses the Nextcloud Text functionality for editing the text on cards. This makes it easier for users to add rich content and supports the new Smart Picker.

Deck also features live editing of boards by multiple people, exporting boards to CSV files and creating new tags when assigning them.

Smart Picker in Deck

Using the Smart Picker in a Deck card. Just type a slash (/).

Performance improvements

In this release, the loading of folder mounts and container handling has been improved by up to 20%. OCS API requests are up to 3x faster and Nextcloud Talk in particular reduced loading times, server load and notification delays with up to 99%.

New Developer Program

As an extension of this release, Nextcloud is introducing a new developer program to super-charge application development. The Nextcloud App Store has already over 300 apps in circulation, however there are countless more apps we’d love to offer and we need your help.

Developer Program

You can work with AI right now

Microsoft just announced its AI tool Copilot for cross-application collaboration, Google its AI in Workspace, but they are only available for a few select customers. Nextcloud’s Smart Picker with AI features is available right now for you to talk to, create with and work with.

And, where Microsoft fired it’s Ethical AI team, we are taking our responsibility serious. We do our best to provide you on-premises solutions where possible, and if that is not possible – we make it clear when a AI solution leaks data to third parties, is not built on an open data set, or not open source.

Let AI assist you in writing a more captivating message, let your words drive your next email or just engage with it to save time on mundane tasks.

This is your Nextcloud. Your opportunity to use AI. Let it work for how you want it to, without compromising your privacy.

A huge thank you to our community! 💙

This is our biggest step forward. A huge thank you to our community for collaborating with us to create the best on-premises collaboration platform in the world! Without you, our user experience would suffer. For all our consumers, the companies, government and public organizations, schools and universities that aspire to progress, thank you for trusting Nextcloud as your collaboration platform. We hope you enjoy all this new release has to offer.