Skip to main content
26,335 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
-3 votes
0 answers
48 views

enter image description here (SQL subquery problem) enter image description here (SQL subquery problem) See the same result came without using the subquery. So why should I use this complicated ...
Saiful Islam's user avatar
  • 1
0 votes
0 answers
46 views

I'm having random data missing/not showing up in BigQuery issues. I would seemingly write data without issues, but SELECT * would return nothing. The control panel shows no errors. None of the quotas ...
Buffalo's user avatar
  • 4,082
0 votes
1 answer
82 views

I'm trying to calculate the percentage contribution of each firm to the total amount across all firms. Database: google-bigquery What I need (desired output): Sum usde_haircut_amt per firm Compute ...
sandesh kamera's user avatar
  • 9
0 votes
0 answers
68 views

I am trying to figure out if there is a way to send a JSON object to a BigQuery table that has a column of type JSON. I know the current practice is to stringify the JSON and send it over which gets ...
Antares's user avatar
  • 100
-6 votes
1 answer
48 views

I’m trying to clean a BigQuery weather dataset where missing values were entered as 0. My UPDATE query to replace 0 with NULL is throwing an error. How can I correctly convert these zeroes to null ...
Saiful Islam's user avatar
  • 1
0 votes
0 answers
28 views

I’m trying to optimize a BigQuery SQL query that joins several large tables. The query works, but it’s slow and more expensive than expected when running on production-scale datasets. Below is a ...
Prathima Sarvani Alla's user avatar
  • 1
1 vote
0 answers
67 views

I am using Databricks Runtime 15.4 (Spark 3.5 / Scala 2.12) on AWS. My goal is to use the latest Google BigQuery connector because I need the direct write method (BigQuery Storage Write API): option(&...
Thilina's user avatar
  • 157
-1 votes
0 answers
35 views

When trying to run a query with a large output (600,000 rows of data), I am getting an error : [Simba}[BigQuery](310) Storage API Error: DNS resolution failed for service: http:. I am using a proxy ...
John Newell's user avatar
  • 1
0 votes
0 answers
24 views

I’m currently working on a task to fetch and display daily Google ADS Manager (GAM) records—such as Cost, ROAS, and other metrics—within a data analysis application. I’ve successfully retrieved data ...
Mehakpal Singh's user avatar
  • 146
0 votes
0 answers
34 views

I work on a project in BigQuery using Looker Studio for dashboard visualizations. We recently saw our billing costs and usage skyrocket and are trying to determine the cause. Nothing fundamentally ...
Brigham Aldrich's user avatar
  • 1
-1 votes
0 answers
65 views

I'm trying to set up connected sheets in Google Sheets, which according to the link below, requires Delegated Access and the access set up in Organization Settings. Is it possible to set these up if I ...
user994165's user avatar
  • 9,562
Advice
0 votes
1 replies
44 views

When i try to do division in bigquery, i saw this. Initial implementation -> div (x / y) for this i got an error : No matching signature for function DIV Argument types: FLOAT64, FLOAT64 Signature: ...
pcbzmani's user avatar
  • 37
0 votes
1 answer
71 views

I’m working on a data ingestion pipeline using Apache Spark (triggered via a Cloud Function on Dataproc). The input CSV contains column names that include special characters such as parentheses and a ...
Suhani Bhatia's user avatar
  • 105
0 votes
1 answer
92 views

I have a BigQuery push subscription in Pub/Sub that for some reason changes the values of the fields with float datatype when pushed to BigQuery. I tried creating a pull subscription and attached it ...
mads.koop's user avatar
  • 1
2 votes
0 answers
47 views

I have a large BigQuery table, big_table, around 5 TB in size. It is partitioned by the column partition_date, which has about 2000 distinct values. I also have a smaller table, small_table, which ...
Alex Lipkin's user avatar
  • 21
1 vote
0 answers
31 views

I’m trying to use the Google Analytics Admin API (v1alpha) to link a GA4 property to BigQuery via the properties.bigQueryLinks.create method. Here is my code using : https://developers.google.com/...
Wendy Longuemare's user avatar
  • 11
0 votes
0 answers
79 views

I'm currently aware that I can export a BigQuery query result to Google Cloud Storage (GCS) by first creating a temporary table and then performing an extract table operation on that temp table. ...
AVA's user avatar
  • 183
0 votes
0 answers
62 views

I have got two BigQuery tables: bigquery:ssh-test-project-01.SSh_Dataset_03.SSh_BgQ_Src_01 and bigquery:ssh-test-project-01.SSh_Dataset_03.SSh_BgQ_Dst_BgQ_01. Using Data Transfer with SQL: "...
Sergey Shabalov's user avatar
  • 23
0 votes
1 answer
64 views

I'm trying to connect a BigQuery table containing property listings (originally from an Excel/CSV file) to a Vertex AI Agent Builder / Dialogflow CX agent using a Structured Data Store. My goal is to ...
John Liko's user avatar
  • 17
-1 votes
3 answers
98 views

I'm trying to run a simple SQL query in BigQuery like this: SELECT usertype, COUNT(*) FROM `project.dataset.table`; But I get an error: SELECT list expression references column usertype which is ...
GAURAVJATT's user avatar
  • 15
1 vote
0 answers
100 views

Some of our BQ projects create external tables on top of parquet files like this: CREATE OR REPLACE EXTERNAL TABLE my_dataset my_table WITH PARTITION COLUMNS (ingestion_date DATE) OPTIONS ( format ...
Etienne Neveu's user avatar
  • 12.7k
0 votes
1 answer
46 views

I have the following code: SELECT h3s.h3id, h3s.geog, MIN(ST_DISTANCE(`carto-os`.carto.H3_CENTER(htsp.h3id), `carto-os`.carto.H3_CENTER(h3s.h3id))) OVER (PARTITION BY h3s.h3id) FROM ...
Chris's user avatar
  • 67
0 votes
0 answers
48 views

I need to define an external table from Google sheet in Dataform - only a few columns from a given sheet should be used. Some AI tools are saying that 'range' can be used in OPTIONS, but I guess it's ...
Sammy Merk's user avatar
  • 19
1 vote
2 answers
192 views

We're a long time users of the Google BigQuery INFORMATION_SCHEMA.TABLE_STORAGE view, never had problem using it in our automated data processes. But a couple of days ago even the simpliest SELECT ...
dreamca4er's user avatar
  • 444
0 votes
0 answers
53 views

I’m using Cloud Composer (Airflow) and have two BigQuery operators like this: run_aggregation = BigQueryInsertJobOperator( task_id='aggregation_task', configuration={ "query":...
efesabanoglu's user avatar
  • 1
0 votes
0 answers
65 views

I'm using dbt models to transform a QBO Balance Sheet uploaded as a CSV into Bigquery. Data comes from a fictional company inside QuickBooks test drives from which I export reports in CSV format. Here'...
almr27's user avatar
  • 53
0 votes
0 answers
46 views

I have configured and put into production a BigQuery environment for one of our customers, and we have a question that we cannot answer, even after numerous analyses on our part. I configured the ...
Luca Magnotta's user avatar
  • 15
1 vote
0 answers
67 views

Problem I have a BigQuery scheduled query that reads from an external table. When I run the query manually, it works perfectly and can access all columns. However, when the same query runs on a ...
Angry birds's user avatar
  • 85
-1 votes
1 answer
88 views

I've installed the Firebase SDK in my iOS app and am using Google Analytics to measure data. The data is exported to BigQuery. The app incorporates consent mode for consent management. I was suddenly ...
engineer's user avatar
  • 1
0 votes
2 answers
180 views

I have a Python pipeline where I try to: get some json data from an API response modify it via duckdb (just in memory) convert the resulting data to a list of python dicts (1 row: 1 dict) Add a ...
Dasph's user avatar
  • 460
0 votes
0 answers
60 views

We use the Firebase Admin Java SDK to send push notifications to iOS devices. Messages appear in the BigQuery firebase_messaging export with message_status = MESSAGE_ACCEPTED, but never progress to ...
Oliver Zhu's user avatar
  • 1
1 vote
2 answers
165 views

I'm new to Dataproc and am having trouble running a job that accesses a PostgreSQL database (Compute Engine VM) to write data to BigQuery. I created a cluster with the following configuration: gcloud ...
rrotter's user avatar
  • 43
0 votes
0 answers
27 views

I am using the table from firebase_crashlytics's export to BigQuery, but some records return a blank user_id, even though our development team has already completed the user_id setup. Firebase only ...
LBH's user avatar
  • 1
3 votes
1 answer
116 views

I have a very large BigQuery table with web events that I want to aggregate into daily, weekly and monthly numbers of visitors. And I want to join them into the same table. My initial guess is to do ...
Jan Mulder's user avatar
  • 35
0 votes
1 answer
99 views

I have the below mentioned BigQuery SQL for a View. However, both the tables used in the query are huge in volume and hence I am facing terrible performance issues. If you'd glance at the query, I am ...
marie20's user avatar
  • 895
0 votes
0 answers
71 views

Trying to write a query which will convert the input, which is list of events with a start and end time as well as the number of apples for that time slot. Example input as is shown below (timings ...
Thijs's user avatar
  • 1
0 votes
1 answer
72 views

My sqlx file config { type: "operations", tags: ["soma_deal_meta_audience_reporting_pipeline"] } ${operations.createSessionizerTable("...
prabodhprakash's user avatar
  • 3,947
1 vote
1 answer
62 views

I ran this standard query and it executes successfully. EXPORT DATA OPTIONS ( uri = 'https://pubsub.googleapis.com/projects/ggdevbigqueryproject/topics/GGTEST', format = 'JSON', ...
dhrubajyoti chatterjee's user avatar
  • 11
-3 votes
1 answer
88 views

WITH CATALOG_SKU_CTE AS ( SELECT A.sku FROM `catalog-stage.catalog_enterprise_views.sku_hierarchy_view` A, `catalog-stage.catalog_enterprise_views.sku_v` B WHERE A.sku = B.SKU AND ...
gcpdev's user avatar
  • 147
0 votes
0 answers
96 views

I'm hoping to get some help with an issue I'm having moving data from a JSON file to a BigQuery table using Spark. The problem, in a nutshell: Goal: Load a JSON file into a Spark DataFrame and persist ...
user3348838's user avatar
  • 1
0 votes
1 answer
119 views

I'm trying to create a job to mirror a view that I have in my PostgreSQL DB to a BigQuery table in my Google Cloud Project through Dataflow, I created the job using the "Job builder", and I'...
Gustavo Trivelatto's user avatar
  • 29
0 votes
0 answers
67 views

I want to reuse the same struct in multiple user defined functions. Is there any way to alias the type so I don't have to write down the definition in full each time? I.e. something like DECLARE TYPE ...
Adam B.'s user avatar
  • 1,190
0 votes
0 answers
71 views

I'm trying to store some data in the google bigquey using nestjs application. I was able to insert few records successfully and after that I changed a schema column. Now I can not insert any records ...
Ajantha Bandara's user avatar
  • 1,551
0 votes
1 answer
125 views

I'm looking for a way to programmatically trigger "Generate Insights for free" for all tables within a specific BigQuery INFORMATION_SCHEMA. Currently, the only method I've found is to ...
Bihag Kashikar's user avatar
  • 1,332
1 vote
0 answers
102 views

I'm working on an ETL tool that can takes data from different sources (eg Snowflake, SQL, etc) and then, in this scenario, insert it into SQL database. While when working with SQL/Snowflake APIs I ...
Alon Albahari's user avatar
  • 163
1 vote
1 answer
72 views

Im using the Google Drive inventory reporting in Google workspace. Im using Apps Script to pull the data into a Google sheet, but the date and time is coming into the sheet in this format: 1.61E+09 ...
Mark Fraher's user avatar
  • 51
0 votes
1 answer
138 views

I'm working with Google BigQuery and I have a table with two numeric columns: grade1 and grade2. I want to calculate the total sum of both columns combined (row-wise) and then find the average of ...
Mahbub Ar Rashid's user avatar
  • 11
1 vote
1 answer
89 views

I'm getting really really inconsistent results, and I can't tell why. Here is the most obvious example: Two queries: select * from my_table where invitation_deleted = true -- returns 118 results ...
matty-d's user avatar
  • 2,669
0 votes
1 answer
70 views

I am unable to create an external connection in BigQuery. I am the Project Owner, but the option "Create connection" is missing from the UI. Expected behavior: When I navigate to the "...
Miro 's user avatar
  • 1
3 votes
1 answer
118 views

We are currently undergoing migration from spark 2.4 to spark 3.5 (and dataproc 1 to 2), and our workflows are failing with the following error Caused by: com.google.cloud.spark.bigquery.repackaged....
Anshul Dubey's user avatar
  • 305

1
2 3 4 5
…
527