Introducing Assistant 2.0

With a nearly complete overhaul of the user interface AND how our AI Assistant works on the back-end, plus a ton of new, big features, we decided that this is the moment to call this the Assistant 2.0. Let’s go over what’s new, but first a short summary of where we are today!

Nextcloud has a ton of AI features, some big and elaborate, others more basic. Some train a neural network on your data, on your own server to give you a smart inbox in Mail or warnings about unexpected logins from Suspicious Login Detection. Others rely on a pre-trained network that can recognize faces and objects in photos, generate a transcript of your video call or modify text. We can also generate images, translate text and chat messages, and much more. Many of the features are accessible through our easy to use Nextcloud Assistant interface!

Many of these features can help you, save you time, or improve your productivity. And best of all: as always we focus on making all these services available of running on your own server. Given even the US Congress doesn’t trust ChatGPT or Microsoft CoPilot, there are good reasons to want control over the data send to and from an AI service! Our Ethical AI Rating system provides you transparency about the AI used to generate your text, image or classify your photos:

• Is it open source? Can I study how it works and re-train or optimize the model for my needs?
• Is the model freely available? Can I run it on my own server?
• Is the training data open and available so I can inspect it for issues and re-train or optimize the model?

Learn more in our blog post about our approach to AI.

Context Chat and Context Write

And now, let’s explore what’s new: The two biggest new features are Context Write and Context Chat. These both share a lot in common, both providing AI-assisted outputs for you, but they’re also distinctly different.

Context Chat, in short, is the ability for the Assistant to answer questions about your data. You can ask the Assistant questions about a document you have: Say, you have a PDF with the manual of your new digital watch… you’ll be able to ask the Assistant how to set it up, or how to replace the battery. And if your company has a nicely documented reimbursement process in its documentation in Collectives, you can ask the Assistant questions about this process. This is an absolutely amazing ability!

It is not just limited to your files either: we developed an API so apps like Mail, Calendar, Talk and Deck can make their data available too. You’ll then be able to ask “When do I have my next meeting with my boss?” (from the Calendar) and “What tasks did she ask me to work on as high priority this week?” (from Mail). Or “Give me a summary of the status of project X.” (Deck + chat perhaps?)

Context Write, on the other hand, lets the Assistant rewrite something in a certain requested context, or style. Say, you have a nice poem and want to write another in that same particular style. By providing the original poem, then new text of what you want to say, the Assistant will then create a masterpiece. But you can also give it a form, give it your data, and ask it to provide the text you would need to fill in the form. Impressive, and useful!

Big user interface overhaul

Over the last year and a half, we have introduced a wide range of AI features in Nextcloud. Translation, transcription of audio, text and image generation, and more. Then the Nextcloud Assistant came – and we’ve now integrated most of our AI features into the Assistant itself, all available from a unified interface. It is no longer limited to text, but can also generate images and transcribe audio.

Another improvement many of you will appreciate is that you can now see a history, a list of earlier generated images, text or audio transcripts in the Assistant interface. You can even go back and generate more based on your earlier prompts. It is now also possible to cancel a running operation, if desired.

Lastly, as possible inputs to the Assistant you can paste or type text or record audio, and now… select one or more documents as input too!

Behind the scenes

We also did significant work behind the scenes. We made it possible for administrators to make use of AI models we’ve not yet explicitly integrated into Nextcloud, giving more flexibility and versatility to your choice of models to use.

Especially for larger deployments, we have made it possible to run the LLM and other AI operations like Transcription and Context Chat on a separate server, using our new external app ecosystem for one-click, Docker-based deployments. You can also now use a GPU for text generation, transcription and context chat. This means transcribing an hour-long audio file could, on a CPU, take many minutes while taking only seconds on a GPU!

More coming soon

And, of course, this is not all. We have developed an API which can be used by apps like Deck, Calendar or Mail to support Context Chat. This will allow the Assistant to answer questions about your mail, tasks or upcoming appointments! We look forward to these and other apps implementing support for this feature.

Nextcloud Hub 8 – our next version out very soon – will also feature two more app integrations of the Assistant: First, Nextcloud Talk will be able to show you summaries of conversation in Talk rooms, and Nextcloud Mail will introduce email ‘suggested answers’! If you haven’t yet registered for our Nextcloud Hub 8 Launch Event on April 24, this is the time to do so to find out more!

How to get it

So now, you ask: When can I get it? We have good news! Nearly all these features have been backported to Nextcloud Hub 7, meaning they are available in the Nextcloud app store right now. All you need to do is update. If you have not yet tried out the assistant, perhaps this is the time to do it!

To run the AI locally, it’s easiest to use LocalAI. A community container exists for our All-in-one, so users of the AIO can simply go to this page to learn more about adding extra containers and then enable the LocalAI container. Be sure to check out our documentation and ask questions in the forums – you could just start below this post!

IONOS, OVHcloud, plusserver and others bring you…

This brings us to the final piece of the puzzle: For these AI features to work smoothly, you will need some serious hardware. This unfortunately will not be a fun experience on your Raspberry Pi. But! Luckily, a number of businesses have started to provide AI-as-a-Service, running various open source models that Nextcloud supports in a way that you can connect to remotely. While this is great, unfortunately nearly all these companies are American-based, with a few in China. If you live in the USA, that is of course perfect – the data stays in your jurisdiction, and you have a choice of local providers. But for our users in Europe, this is less than ideal. The US jurisdiction unfortunately treats the data of European users with less than ideal respect for privacy, in no small part thanks to the CLOUD act signed into law by the former US president Donald Trump.

But today, we announce that we have been working closely with several major European hosting companies including IONOS, OVHcloud and plusserver to integrate their upcoming AI-as-a-Service solutions. This will make them available to Nextcloud customers and users who wish to take advantage of the advanced AI functionalities in Nextcloud Hub but don’t wish to host AI services themselves. Several Nextcloud customers have expressed interest in these features and are preparing to or in the process of testing them. Here’s what our partners and customers have to say:

OVHcloud is thrilled to collaborate with Nextcloud in delivering state-of-the-art Digital Sovereign AI solutions to our customers. We are aligned with Nextcloud’s commitment to offering secure, privacy-conscious AI functionalities that safeguard training data and adhere strictly to GDPR regulations. We are excited to announce that this integration will become available in Q2 2024, marking a significant milestone in our journey towards responsible AI innovation.

Germain Masse, AI Product Marketing Manager at OVHCloud

We are proud to offer Nextcloud customers a solution that complies with European data protection regulations. Reliable, trustworthy, and local AI services will be key to protecting their digital sovereignty for both the public and private sectors in Europe.

IONOS

plusKI brings cutting edge, open source AI technologies in a compliant and digitally sovereign service to the German market. We are excited to work with Nextcloud to make this available to their customers as back-end to the new generation of the Nextcloud Assistant.

Christian Schmitz, Director Open Source at plusserver

We at SUNET are happy that that Nextcloud Assistant is 100% Free and Open Source, runs on-premise and provides useful features that we hope will benefit our users going forward. We enjoy the very productive collaboration with Nextcloud.

Mike Nordin, Sunet, the Swedish University Computer Network

In the future, we will make AI services available to colleagues in the state administration directly at the workplace. We want to drive the development of digitally sovereign and open solutions in addition to the use of existing AI services such as ChatGPT. Because, the administration of the future will work in an automated, algorithmized, cloudified and data-based way. To make this vision of the future a reality, we must provide the appropriate tools. This also creates value and jobs in our domestic digital economy.

Minister of Digitisation and Head of the State Chancellery Dirk Schrödter, Schleswig-Holstein, Germany

US technology firms have a large head start on European companies offering large language models and other AI services to smaller firms and end customers. There is a significant number of offerings either using their own models like OpenAI, or providing one or a choice of open models like Llama, Gemma or Mistral as a service. In Europe, Mistral and Aleph Alpha are still working on offering their own models while only a few large hosting providers are looking to offer various open LLM’s in an AI-as-a-service model.

At the same time, there is a growing demand from our customers and users for European AI services. Running AI locally is expensive, and AI-as-a-Service from foreign tech firms is not a solution for most Nextcloud customers in Europe, given the privacy and sovereignty risks inherent to shipping large amounts of crucial data overseas. For this reason, we have been working with a handful of providers to offer their AI services. To speed up the process, we connected them with potential customers. As you can imagine, offering a service when it is unclear who its customers might be is not easy. But several universities are interested in the ability to outsource some heavy AI operations, so there is now movement towards AI services. This means not only our users, but the wider European ecosystem will be able to benefit from this!

Public availability of the European AI services is expected in the course of spring and summer, with some even aiming for late April/early May!

Thank you!

We are super excited about this release and we’d like to thank everybody who contributed, with testing, reviewing code but also the universities who are willing to test the AI services, and of course OVHcloud, IONOS, plusserver and our other partners who’d like to provide these services!

The post Nextcloud releases Assistant 2.0 and pushes AI-as-a-Service appeared first on Nextcloud.

Last Wednesday, social media buzzed with apprehension over Dropbox’s new AI feature that shares data with OpenAI to perform an experimental AI-powered universal search across user data on the platform. As reported by Ars Technica and other media, the on-by-default setting led to understandable upset from many Dropbox users.

While Dropbox does offer the option to opt-out of their new feature, something that sends your data to a third party company shouldn’t be on by default without first checking with the user if they’re okay with it. At the very least, they could have enabled a pop-up dialog the first time they showed the feature, explaining the consequences and offering that opt-out in an easy way.

When we at Nextcloud introduced our AI feature set in early 2023, we recognized the challenges intrinsic to Artificial Intelligence, and from day 1 have prioritized total transparency, choice, availability to users, and the honoring of adjoining ethical implications. Nextcloud Hub features rich AI offerings with an always privacy-first approach: while many are available, our remote-AI features are optional and disabled-by-default, never exposing user data without your knowledge or consent.

For example, we released an article recently that reiterates our AI approach and explains the difference between our completely self-hosted AI options versus those implemented as integrations from external parties. We then dove into each new AI feature to illustrate their utility, the different options to enable them, and any potential AI and privacy-related implications, aided by our Ethical AI Rating system.

In Nextcloud Hub 7, we just released our advanced, Unified Search that allows you to search for any file, person and more in Nextcloud. Since the data is only indexed on your server, no data leaves your server. Nextcloud can store all your data from contacts to documents and notes, so you don’t even have a need for using or interacting with a 3^rd party service if you don’t want to.

Similarly, the Nextcloud AI Assistant uses the collective data from your server to help you enhance and speed up your work with all-local AI options. You can get language assistance in your documents, summarize your email threads and Talk chats, and more.

We just shared a tech preview of our Nextcloud AI Assistant with Context Chat, a new initiative developed together with the German state Schleswig-Holstein that enables local AI models within Nextcloud to search users’ local data across the platform. While on the surface this appear similar to Dropbox Dash – their new cross-platform search utility – the key implication and difference is in the implementation: Dropbox relies solely on sending user data to OpenAI for AI-assisted processing, while Nextcloud’s Assistant with Context Chat can rely completely on in-house, self-hosted local AI models, a real privacy [boon].

What we are illustrating here is that yes, AI can give great productivity benefits. And you can bring that to users in a responsible way. Even if you’re not willing or able to offer AI features in-house or on-premises and instead want to rely on OpenAI or other big AI companies, there is no need for sharing your users’ data to such a third party without a warning.

Thank you for reading and staying up-to-date about current data privacy news that we, like you, care deeply about. We hope you find Nextcloud solves your needs and provides you unmatched opportunities to use and work with AI securely, easily, and privately. We would also love to continue the conversation: send us your thoughts via Mastodon and X (Twitter).

The post Dropbox handing data to OpenAI: the risks of AI in practice appeared first on Nextcloud.