419 questions
0
votes
1
answer
43
views
Replace value inplace while saving the old value
I'm looking to use yq to replace a yaml file values while still retaining the old value.
Essentially putting base/1.0.0/ after base.routes.paths ~/ and what comes after. The added value will be the ...
- 840
1
vote
1
answer
84
views
YQ: load a text file as array and use it for an operation
I have a yaml file with an arbitrary amount of documents, and I'm trying to replace all missing namespaces for namespaceable resources with an arbitrary input one.
Getting the non-namespaceable ...
- 809
0
votes
0
answers
88
views
Avoid encoding emojis when using yq --prettyPrint
When I convert my JSON file to YAML, i want any unicode chars to be part of the content and not escaped with backslash and quotes. Example: This is my file.json
What can I do in order to make YQ not ...
- 3,132
0
votes
2
answers
73
views
Extract and merge object contents from two different trees
I have the following structure in yaml:
root1:
foo:
prop1: val1
bar:
prop1: val2
root2:
foo:
prop2: val3
bar:
prop2: val4
I wish to split the tree in multiple docs along the ...
- 39
2
votes
1
answer
91
views
How to change a specific YAML value for only certain YAML objects with yq
Let's assume the following YAML file:
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: Bucket
metadata:
name: kratix
namespace: flux-system
spec:
interval: 10s
provider: generic
...
- 2,297
1
vote
2
answers
89
views
using mikefarah/yq find the Nested keys with parent
need to get all the nested keys from a yaml file using mikefarah/yq(installed v4.25.1)
tried the below 2
this gives only the parent level keys not the child keys under it
yq eval 'keys' "...
- 189
0
votes
1
answer
111
views
How to update the values for all keys matching a given pattern using yq?
I have a yaml file looks like
part1:
name: tom
some_field: true
part2:
name: jerry
other_field: true
and I would like to change the value of all keys that end with field into false, that is, ...
- 24.9k
0
votes
2
answers
142
views
How do I handle child objects properly with jq map()
I'm currently trying to build a pipeline to process Windows .evtx binary logs into usable JSON for easy searching and filtering on Linux, but due to going through xml, there is data that needs to be ...
- 13
0
votes
1
answer
98
views
Apply jq functions conditionally to matching and non-matching JSON subtrees
I want to apply one function to all descendants of a particular node in a tree, and a different function to all other nodes, in the following way:
if path_matches(.deep.path.matching) then ...
- 1,441
-3
votes
1
answer
80
views
Array in YAML can't be used
I have these variables in a YAML format that I enter from the pipeline (see image). I'd like to extract them and define them as env var using this command when running the pipeline.
Pipeline variables ...
- 5
0
votes
1
answer
59
views
How to replace forbidden characters in dynamic target filename of yq split
I want to split a k8s resource file with mikefarah/yq into individual files per resource. Obviously the target filename of the new files have to be dynamic.
A quite obvious naming-scheme for k8s ...
- 7,067
0
votes
1
answer
73
views
yq expression to replace first-level keys with their values?
Given input JSON like this:
{
"a": {
"aa": {
"aaa1": "value",
"aaa2": "value"
}
},
"...
- 2,143
1
vote
1
answer
293
views
Use yq to add to a YAML file with anchors, aliases and arrays
I have a YAML file like this:
keys:
- &node0001 ageXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
- &node0002 ageYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
...
0
votes
1
answer
47
views
yq expression to prune "dead" branches from JSON?
Given input JSON that is "sparsely populated" with values, such as:
{
"a": {
"b": {},
"c": {
"d": {
...
- 2,143
1
vote
1
answer
85
views
yq: compose new key/value depending on value of other keys
This is the input:
- hostname: rk089-s11
compute: false
control: true
master: false
- hostname: rk090-s11
compute: false
control: true
master: true
I am trying to get this output:
- ...
- 2,602
0
votes
1
answer
444
views
How to use yq on multiple documents, selected by content not by index
If I have multiple documents in one file (example.yaml), like in a k8s manifest:
kind: Service
---
kind: Deployment
Is there a way I can select documents other than by their numeric index?
My final ...
- 3
0
votes
1
answer
57
views
Why do I get twice the same JSON output with mikefarah/yq?
I have installed mikefarah/yq :
$ yq -V
yq (https://github.com/mikefarah/yq/) version v4.44.3
$
I want to create a JSON structure with yq.
I tried this yq command :
yq -n -o=json '.a = 1,.b.c = 16 , ....
- 4,965
0
votes
1
answer
71
views
How to append proxies items to an existing /etc/docker/daemon.json file with yq
I have the following /etc/docker/daemon.json file :
$ yq . /etc/docker/daemon.json
{
"dns": [
"a.b.c.d1"
]
}
$
I tried this command to append proxy lines to this file with ...
- 4,965
0
votes
0
answers
242
views
Append new lines to string in yaml with yq
I am trying to append a following Variable to my yaml
GROUPS="\ng, ZI-LNX_1, role:metadata-viewer \ng, ZI-LNX_2, role:metadata-viewer \n"
Yaml looks like this
...
rbac:
...
- 404
0
votes
1
answer
42
views
yq is not properly interpreting map from list of maps if it comes from env variable
I am trying to append new entries to the following yaml:
destinations:
- namespace: ns1
server: https://kubernetes.default.svc
What I expect is:
destinations:
- namespace: ns1
...
- 404
0
votes
1
answer
135
views
How to remove empty yaml properties?
I need to remove all properties of yaml in markdown files that have empty values recursively in a directory.
Before:
---
aliases:
cssclasses:
- rss
tags:
---
After:
---
cssclasses:
- rss
---
It ...
- 71
0
votes
1
answer
65
views
Updating key on all child values with yq
I have a YAML like:
foo:
thing_1:
key_to_update: "a"
thing_2:
key_to_update: "b"
(etc)
How do I do foo[*].key_to_update = "x", effectively setting ...
- 4,457
0
votes
1
answer
700
views
How to merge two yaml files while overriding duplicates using yq4
I have two files. The first file is a file where all the defaults are set.
defauly_deploy.yaml
deploy:
- env: prod
disabled: true
namespaces:
- production
disable_switch:
a: ...
- 314
0
votes
0
answers
61
views
Update multiple nodes using YQ
Istio recently introduced a CEL validation rule that conflicts with our custom go templating engine return errors such as this:
{"GenericComponent":{"name":"istio","...
- 61
0
votes
1
answer
383
views
yq wont recognize single quotation marks around the field when trying to split large YAML file into smaller ones
I have a large YAML file containing info about video games.
---
-
Name: Metal Gear Solid HD Collection
Console: PS3
Genre: Stealth
Progression: Unplayed
Maintenance: Perfect!
-
...
0
votes
1
answer
40
views
Map key to its child value attribute
I have a simple YAML:
host1:
type: A
value: 10.0.0.1
host2:
type: A
value: 10.0.0.2
which I want to map key-value pairs, like this:
host1: 10.0.0.1
host2: 10.0.0.2
How can I do this using yq?...
- 32.9k
0
votes
1
answer
52
views
Sort kubernetes NetworkPolicy by ports and then by ipBlock using yq
I have the following kubernetes NetworkPolicy file:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
acm_host_group: 10.20.30.40/27,11.20.30.40/27
...
- 357
0
votes
0
answers
489
views
yq utility fails with error: unknown shorthand flag: 'c' in -c
I am trying to use yq utility to parse for validation values.yaml files, the resources block in particular. Below is a simple gitlab ci pipeline that I created to test yq utility
stages:
- test
...
- 107
0
votes
1
answer
412
views
Adding items with default values to empty array with yq (go)
ok I've been in contact with chatgpt and gemmini on this and they are really out to lunch a lot of the time.
I am using https://github.com/mikefarah/yq
I am reading from xml and need to ensure single ...
- 471
1
vote
1
answer
3k
views
YQ - how to filter an array by field value if matching at least one element in a list
Specifically (https://github.com/mikefarah/yq) 4.35+
I have a comma-separated string of terms and I want to filter down a list if a field value is one of those terms.
Ex) Given the following input:
- ...
- 23
0
votes
1
answer
1k
views
Add item to list in yq if it doesn't already exist
Using mikefarah's yq v4 I want to conditionally add an IP address to a list within a yml file if it does not exist in the list already.
Given a file like:
servers:
web:
- 1.1.1.1
jobs:
- 2....
- 5,656
1
vote
1
answer
168
views
Simple yq (go) to join key values and output to markdown
I am trying to use the go version of yq to produce markdown from a json array.
Given the following json array:
[
{ "en": "cat", "fr": "chat" },
{ "...
- 471
0
votes
1
answer
76
views
Adding values to YAML file
`
I want to append an entry in namespaceAdmin,namespaceDebugger in the below yaml file using yq. Basically it has got multiple patches and I want to make changes in some of the helmrelease patches. Is ...
- 1
1
vote
0
answers
843
views
How do I use `yq` to count the number of yaml documents present in multidocument yaml file that uses the `---` separator?
Given yaml file:
---
one: hey
---
two: you
---
How do I use yq to count the number of distinct yaml documents in the file?
yq ea '. as $item ireduce (0; . + 1)'
yields
3
3
3
and
yq e '. as $item ...
- 2,207
-2
votes
1
answer
455
views
how to use yq with ansible to generate quotes around inserted items?
I am trying to add some items in an YAML file, calling yq (v4.43.1) from ansible (core 2.16.5).
It inserts the values correctly, but without the quotes that would make it a valid yaml file.
Now I am ...
1
vote
1
answer
175
views
going back to 1 document after `.[]` in yq
Case 1
With input
- groupx:
- groupy:
The yq expression length evals to 2
Case 2
With input
groupx:
groupy:
The yq expression length also evals to 2
Case 3 - using splat (.[]) to transform case1 to ...
- 4,413
0
votes
1
answer
581
views
Conditional statement in yq parser for updating script
I m trying to replace a string value in a yaml file and I m not able to get the if working.
YAML file
apiVersion: kots.io/v1beta1
kind: Config
metadata:
name: enterprise
spec:
groups:
- name: ...
- 2,144
1
vote
1
answer
1k
views
how to read a yaml file in bash shell and iterate through a list inside
I have this yaml file
I have a yaml file as below:
repos:
- REPO_NAME: A
INGRESS_SUBDOMAIN: B
DOCKER_IMAGE_PATH: C
IMAGE_NAME: D
MAJOR_VERSION: E
MINOR_VERSION: F
the name of the file is f....
- 2,565
0
votes
1
answer
622
views
yq deal with list items that can be values or objects
I'm trying to do some pipeline work that injects certain values in a yaml document (an mkdocs config) if they are not currently present. But the problem is that the current values can be either items ...
- 179
0
votes
1
answer
596
views
Delete an entry with the key matching regex using yq
I have a json like this:
{
"sha256key": [
"438143050f234146300c44ed7e22be698e1e3c8c6bbc89b5c7e6276e8250a3a7"
],
"validkey": [
"...
- 109
1
vote
1
answer
171
views
yq traverse complex values and convert to json file
I have a complex yaml and want to extract information from it using bash script.
The yaml is like:
content:
images:
sha256:4c8b96d4fffdfae29258d94a22ae4ad1fe36139d47288b8960d9958d1e63a9d0:
...
- 109
0
votes
1
answer
126
views
Inverse operation for a yaml merge such as yq multiply?
Many application read several input files and merge them logically such that inputs with higher priority override previously read fields.
For yaml files, we can achieve this logic using a merge ...
- 731
2
votes
1
answer
134
views
How to merge unique values for a child array when merging objects
I want to merge the .environments.default and .environments.dev objects together, so that dev values override any default values and default values fill in anything not defined in the dev object. If I ...
- 33
0
votes
1
answer
666
views
read all values from multiple documents within a single file into an array with yq
I use mikefahra's yq 4.40.5 and try to read values from a files which may contain multiple documents into one array.
I want to get all .port and .nodePort fields into one array which is contained in ...
- 2,103
0
votes
1
answer
4k
views
How do I build a YAML list using yq?
I am trying to use Mike Farrah's yq utility to build YAML files from within a bash script. Below is an example of the template I'm using:
---
name: <name>
description: <description>
type: &...
- 2,297
-1
votes
1
answer
1k
views
How use yq to transform list format
Given following yaml format
value:
- 1
- 2
- 3
How to use yq to output
value: [ 1, 2, 3]
- 43
0
votes
1
answer
938
views
YQ update style of key/value attribute
On YQ v4, i would need to transform a yaml file like:
test:
A: "customData"
B: 123
Into a .env format :
A="customData"
B=123
I have come up with this command:
yq '.test | ...
- 1,141
0
votes
1
answer
361
views
Is it possible to do a inverse match using yq
i wanted to do inverse match of a yml as below
Input yml content :
....
....
validate-common-non-production:
stage: validate-common-non-production
deploy-common-non-production:
stage: deploy-...
- 21
0
votes
1
answer
218
views
yq command to remove unnecessary fields
Given a yaml file, I am looking for a yq command that will output a more "clean" list with only the relevant key "name" and its value.
for example, an output of the following yaml:
...
- 1,425
0
votes
1
answer
227
views
yq inline replace removes the rest of the file
I have the following yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-backend
labels:
app: app-backend
spec:
replicas: 1
selector:
matchLabels:
app: app-backend
...
- 381