1

So I am working on a project using proto buff where I need to generate golang code using .proto files. My issue is that it is importing wrong package when I generate golang code.

So this is how my project structure looks:-

myproject/
├── protos/
│   ├── models/
│   │   ├── ad.proto
│   │   └── enums.proto
│   └── requests/
│       └── ad_request.proto

The "ad_request.proto" file requires some models or messages from "ad.proto" and "enums.proto" files For that I imported these files and used those messages like this-

syntax = "proto3";

package my_project.requests;

import "models/ad.proto";
import "models/enums.proto";

option go_package = "github.com/name/my_project/protos/requests;requests";

so now after I generate code, a go file is made - "ad_requests.pb.go"

But the problem is that it is importing wrong package
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
//  protoc-gen-go v1.28.1
//  protoc        (unknown)
// source: requests/ad_request.proto

package requests

import (
    models "github.com/name/my_project/protos/models"
    protoreflect "google.golang.org/protobuf/reflect/protoreflect"
    protoimpl "google.golang.org/protobuf/runtime/protoimpl"
    reflect "reflect"
    sync "sync"
)

instead of importing github.com/name/my_project/models it is importing file from proto folder but there is no golang files in proto folder

Anyone know how to resolve this issue

This is some info on ad.proto

syntax = "proto3";

package my_project.models;

import "google/protobuf/timestamp.proto";
import "models/enums.proto";

option go_package = "github.com/name/my_project/protos/models;models";

This is the make command I use to auto generate code


.PHONY: gen proto-gen
gen proto-gen:
    buf generate protos --template protos/buf.gen.yaml
    protoc-go-inject-tag -input=./internal/models/*.pb.go -remove_tag_comment
    protoc-go-inject-tag -input=./internal/requests/*.pb.go -remove_tag_comment
3
  • What protoc command are you using? Commented Apr 30, 2024 at 14:55
  • @DazWilkin Using make command I am executing this- gen proto-gen: buf generate protos --template protos/buf.gen.yaml \n protoc-go-inject-tag -input=./internal/models/*.pb.go -remove_tag_comment \n protoc-go-inject-tag -input=./internal/requests/*.pb.go -remove_tag_comment \n Commented Apr 30, 2024 at 15:09
  • You have the change the option go_package lines to point to the right place, where you want to do the imports from. So option go_package = github.com/name/my_project/models for example. Commented May 1, 2024 at 21:38

2 Answers 2

1

From your Makefile I assume you want your generated files in github.com/name/my_project/internal (since you have -input=./internal/models/*.pb.go).

Your protos/buf.gen.yaml would be something like:

version: v1
plugins:
  - plugin: buf.build/protocolbuffers/go:v1.34.0
    out: internal
    opt:
      - paths=source_relative

protos/requests/ad_request.proto would be:

syntax = "proto3";

package my_project.requests;

import "models/ad.proto";
import "models/enums.proto";

option go_package = "github.com/name/my_project/internal/requests";

message Msg {
  optional my_project.models.Enum enum = 1;
}

and protos/models/enums.proto:

syntax = "proto3";

package my_project.models;

option go_package = "github.com/name/my_project/internal/models";

enum Enum {
  A = 0;
}

Then you could do:

package main

import (
    "fmt"

    "github.com/name/my_project/internal/models"
    "github.com/name/my_project/internal/requests"
)

func main() {
    msg := &requests.Msg{Enum: models.Enum_A.Enum()}

    fmt.Println(msg)
}

assuming you have a go.mod:

module github.com/name/my_project

go 1.22.2

require google.golang.org/protobuf v1.34.0

A. side note on conventions: but.gen.yaml should be at the root of the project and the usage of ; in go_package is discouraged.

Sign up to request clarification or add additional context in comments.

Comments

0

You added your comment after I started writing this.

I don't use buf so apologies that this is pure protoc.

I'm assuming you want to generate the Go sources alongside the protos.

The Protocol Buffers package hierarchy does not translate directly to the Golang module hierarchy. Your approach to using option go_package is correct but I would advise against using e.g. ...models;models"

Here's how I would generate your protos. I use the --go_opt=module=${MODULE} approach which works for me. There are other ways to achieve this.

NOTE If you subsequently want to compile Services for gRPC, you would mirror this approach by adding e.g. ... --go-gprc_out=${PWD} --go-grpc_opt=module=${MODULE} ...

In this approach ${MODULE} must match the prefix (as you have) for your options go_package={MODULE}/protos/...

Either:

MODULE="github.com/name/my_project"

PROTO_PATH=${PWD}/protos

protoc \
--proto_path=${PROTO_PATH} \
--go_out=${PWD} \
--go_opt=module=${MODULE} \
${PROTO_PATH}/requests/ad_requests.proto \
${PROTO_PATH}/models/ad.proto
${PROTO_PATH}/models/enums.proto

The rule here is that every *.proto must be prefixed with one (you only have one) of the defined --proto_path's

One caveat (but not bug) in your approach is that your Protocol Buffers packages should be models and requests and not my_project.models and my_project.requests.

If you want these to be e.g. package my_project.models then you should create:

.
└── protos
    └── my_project
        ├── models
        └── requests

This is because your current folder's protos is the root of your protocol buffers files. The convention (!?) is to then reproduce the package path as a folder structure. Because you start with my_project, the first folder within protos would be my_project etc.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.