Evaluation

Real-world Evaluation (infilling)

masking out random snippets in code with cross-file dependencies, generating samples from the model, and retesting the code files with the generated snippets; because of recently committed open-source code, an internal version with the same testing methodology is used; model also tested within live coding environments against current Google completion models