1

Git LFS doesn't work anymore with Yocto: git doesn't fetch "large files" since we upgraded our tools to these versions:

Yocto kirkstone
gitlab 16.7.3 and gitlab 16.8
git version 2.34.1
git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
Kubuntu 22.04: Linux linux-dev 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

When I execute the Yocto command "bitbake tep2020ihm", I get multiple "Smudge" errors, like that one:

ERROR: tep2020ihm-1.22-r0 do_unpack: Bitbake Fetcher Error: FetchError( <...>) failed with exit code 128, output:\nDownloading 3rd-party/aghaf/lib/aghaf.dll (6.6 MB)\nError downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll

When I'm cloning the project manually (to another directory on the dev server, by typing manually the command "git clone https://...", it works, including the large files. When the Yocto fetcher does it, it does not work (smudge errors for large files).

If I put the mention "lfs=0" in the SRC_URI variable of the .bb file (the Yocto recipe for my project), the fetcher works, but instead of fetching the file's content, it fetches the "LFS blob" instead:

version https://git-lfs.github.com/spec/v1
oid sha256:dba4dab327a3b420509ac9f466acde6c657698ccb1854522d20d3cfb35a493c1
size 173735936

I'm producing below the result of the command "git lfs logs last" in case it may clarify a point that I forgot to mention:

git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
git version 2.34.1

$ git-lfs filter-process
Error downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd): batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution

batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd)
github.com/git-lfs/git-lfs/errors.newWrappedError
        github.com/git-lfs/git-lfs/errors/types.go:225
github.com/git-lfs/git-lfs/errors.Wrapf
        github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
        github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:119
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
        github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:78
github.com/git-lfs/git-lfs/commands.smudge
        github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
        github.com/git-lfs/git-lfs/commands/command_filter_process.go:122
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/cobra/command.go:902
github.com/git-lfs/git-lfs/commands.Run
        github.com/git-lfs/git-lfs/commands/run.go:105
main.main
        github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
        runtime/proc.go:250
runtime.goexit
        runtime/asm_amd64.s:1571
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
        github.com/git-lfs/git-lfs/errors/types.go:225
github.com/git-lfs/git-lfs/errors.NewSmudgeError
        github.com/git-lfs/git-lfs/errors/types.go:311
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
        github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:87
github.com/git-lfs/git-lfs/commands.smudge
        github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
        github.com/git-lfs/git-lfs/commands/command_filter_process.go:122
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/cobra/command.go:902
github.com/git-lfs/git-lfs/commands.Run
        github.com/git-lfs/git-lfs/commands/run.go:105
main.main
        github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
        runtime/proc.go:250
runtime.goexit
        runtime/asm_amd64.s:1571

Current time in UTC: 
2024-01-18 17:17:38

ENV:
LocalWorkingDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git
LocalGitDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git
LocalGitStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git
LocalMediaDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/objects
LocalReferenceDirs=
TempDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/tmp
ConcurrentTransfers=8
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic,lfs-standalone-file,ssh
UploadTransfers=basic,lfs-standalone-file,ssh
GIT_EXEC_PATH=/usr/lib/git-core
GIT_CEILING_DIRECTORIES=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0
GIT_PREFIX=
GIT_CONFIG_PARAMETERS='core.fsyncobjectfiles'='0' 'gc.autoDetach'='false' 'core.pager'='cat'

Client IP addresses:

1 Answer 1

1
+50

There is a bug in yocto git fetcher which make that nobranch is incompatible with lfs option. you have to use the branch=master and lfs=1 to make it work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.