A 770B model on premises: what are you actually sizing?

If you want certain documents to stay within your company, running a model locally may make sense. The next question is what task it must solve and with what resources. Choosing the largest model first can lead you to size a system you do not need.

Tencent announced Hy4 preview on 28 August 2026 with 770 billion total parameters, 49 billion active parameters and a context of over one million tokens. These are manufacturer specifications. They do not establish the speed you will get or the infrastructure your application needs.

Weights are only part of the memory

As a theoretical calculation, storing 770 billion values at one byte per value amounts to 770 billion bytes: 770 decimal GB. At four bits per value, it would be 385 decimal GB. These are calculations for representing weights; they are not the service's total memory or verified sizes of a specific Hy4 distribution.

Actual quantisation may add scales, metadata and components that retain another precision. You must also check whether the engine supports that version of the model. Having only some parameters active in a calculation step does not mean you only need to store that portion of the weights.

Inference requires additional resources. In architectures that use a KV cache, it retains information to generate subsequent tokens and can consume a lot of memory with long contexts. Hugging Face explains cache options, offloading to CPU and quantisation, with performance trade-offs. That documentation is not a deployment recipe for Hy4.

Context length, simultaneous conversations, the engine and distribution across devices change the requirements. I would not turn the 770 GB into a shopping list of cards or a budget. You first need a compatible configuration and measurements of peak memory and response times under the expected load.

A small task can justify local deployment

In a hypothetical case, a professional practice wants to extract a due date and a reference from internal documents. I would prepare representative examples, including documents without those fields and contradictory versions. The expected answer should provide the information and where it appears, or acknowledge that it is missing.

I would test a smaller model and another candidate with the same collection. I would measure accuracy, omissions, invented answers, seconds to obtain a result and minutes spent reviewing it. A small model may be sufficient or fall short; the trial decides. For questions about documentation, I would also compare conventional search before adding generation.

Local deployment lets you define where the service runs, but it still needs permissions, updates, logs and backups. I would check whether any part of the application sends data outside, beyond the model: monitoring, connected tools or support. “Local” should describe the data's journey.

Choose an operation you can sustain

I would compare local, hosted and hybrid services by quality, permitted data, capacity and maintenance. For an external service, I would review processing, retention and access conditions; for a local one, who maintains it and how it recovers from failure. Choosing a label does not provide a general privacy guarantee.

For a focused task, I would neither rule out local deployment on principle nor assume that 770B is the right choice. I would ask for a trial with acceptance criteria before committing to infrastructure. The result you need is more specific than the model's size.