Skip to main content
26,335 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
-7 votes
1 answer
50 views

I’m trying to clean a BigQuery weather dataset where missing values were entered as 0. My UPDATE query to replace 0 with NULL is throwing an error. How can I correctly convert these zeroes to null ...
Dale K's user avatar
  • 28.1k
-3 votes
0 answers
58 views

enter image description here (SQL subquery problem) enter image description here (SQL subquery problem) See the same result came without using the subquery. So why should I use this complicated ...
marc_s's user avatar
  • 760k
0 votes
2 answers
4k views

I have a BigQuery stored procedure that is defined like this: CREATE OR REPLACE PROCEDURE proc_name( param_1 INT64, param_2 INT64, param_3 DATE ) BEGIN //CODE END When I call the ...
Michal Kac's user avatar
  • 1
1 vote
2 answers
316 views

I'd like to concatenate tables with different schemas, filling unknown values with null. Simply using UNION ALL of course does not work like this: WITH x AS (SELECT 1 AS a, 2 AS b ), y AS (SELECT ...
Tobias Hermann's user avatar
  • 11.2k
0 votes
0 answers
48 views

I'm having random data missing/not showing up in BigQuery issues. I would seemingly write data without issues, but SELECT * would return nothing. The control panel shows no errors. None of the quotas ...
Buffalo's user avatar
  • 4,082
0 votes
2 answers
165 views

I'm trying to create some reusable functions within my SQLX file and I'm finding that if I use CREATE TEMP FUNCTION statements, that it fails when I try to execute the job. Take this very basic ...
Matt Scheurich's user avatar
  • 1,089
4 votes
2 answers
9k views

ARRAY_TO_STRING() works with string arrays but is not supported with integer arrays. I have a repeated column that contains a list of IDs, I want to export this from a US dataset to an EU dataset. So ...
Lukas Eder's user avatar
  • 223k
0 votes
1 answer
83 views

I'm trying to calculate the percentage contribution of each firm to the total amount across all firms. Database: google-bigquery What I need (desired output): Sum usde_haircut_amt per firm Compute ...
Dale K's user avatar
  • 28.1k
0 votes
0 answers
70 views

I am trying to figure out if there is a way to send a JSON object to a BigQuery table that has a column of type JSON. I know the current practice is to stringify the JSON and send it over which gets ...
Antares's user avatar
  • 100
0 votes
1 answer
64 views

I'm trying to connect a BigQuery table containing property listings (originally from an Excel/CSV file) to a Vertex AI Agent Builder / Dialogflow CX agent using a Structured Data Store. My goal is to ...
John Liko's user avatar
  • 17
0 votes
0 answers
29 views

I’m trying to optimize a BigQuery SQL query that joins several large tables. The query works, but it’s slow and more expensive than expected when running on production-scale datasets. Below is a ...
Prathima Sarvani Alla's user avatar
  • 1
1 vote
0 answers
67 views

I am using Databricks Runtime 15.4 (Spark 3.5 / Scala 2.12) on AWS. My goal is to use the latest Google BigQuery connector because I need the direct write method (BigQuery Storage Write API): option(&...
Thilina's user avatar
  • 157
-1 votes
0 answers
35 views

When trying to run a query with a large output (600,000 rows of data), I am getting an error : [Simba}[BigQuery](310) Storage API Error: DNS resolution failed for service: http:. I am using a proxy ...
John Newell's user avatar
  • 1
0 votes
0 answers
24 views

I’m currently working on a task to fetch and display daily Google ADS Manager (GAM) records—such as Cost, ROAS, and other metrics—within a data analysis application. I’ve successfully retrieved data ...
Mehakpal Singh's user avatar
  • 146
0 votes
0 answers
34 views

I work on a project in BigQuery using Looker Studio for dashboard visualizations. We recently saw our billing costs and usage skyrocket and are trying to determine the cause. Nothing fundamentally ...
jarlh's user avatar
  • 44.9k
0 votes
1 answer
92 views

I have a BigQuery push subscription in Pub/Sub that for some reason changes the values of the fields with float datatype when pushed to BigQuery. I tried creating a pull subscription and attached it ...
Kamal Aboul-Hosn's user avatar
  • 17.5k
-1 votes
0 answers
65 views

I'm trying to set up connected sheets in Google Sheets, which according to the link below, requires Delegated Access and the access set up in Organization Settings. Is it possible to set these up if I ...
Johns Cherian's user avatar
  • 1
1 vote
2 answers
3k views

I have created models in my dbt(data build tool) where I have specified column description. In my dbt_project.yml file as shown below models: sakila_dbt_project: # Applies to all files under ...
Juergen's user avatar
  • 352
2 votes
2 answers
2k views

I am creating a BigQuery table using the Python API. I want to make user_id as the primary key. How do I add this constraint using the Python API? from google.cloud import bigquery def create_table(): ...
igi79's user avatar
  • 1
0 votes
2 answers
4k views

I am trying to force dbt to skip a model if source table was not found, but I didn't found any resources that could help on such topic, for more detail on the context, I have a folder called sales on ...
KiranM's user avatar
  • 1,324
Advice
0 votes
1 replies
44 views

When i try to do division in bigquery, i saw this. Initial implementation -> div (x / y) for this i got an error : No matching signature for function DIV Argument types: FLOAT64, FLOAT64 Signature: ...
Joop Eggen's user avatar
  • 110k
0 votes
1 answer
71 views

I’m working on a data ingestion pipeline using Apache Spark (triggered via a Cloud Function on Dataproc). The input CSV contains column names that include special characters such as parentheses and a ...
Frank's user avatar
  • 634
2 votes
0 answers
47 views

I have a large BigQuery table, big_table, around 5 TB in size. It is partitioned by the column partition_date, which has about 2000 distinct values. I also have a smaller table, small_table, which ...
Alparslan ŞEN's user avatar
  • 688
2 votes
1 answer
334 views

I need to write a polars dataframe into a BigQuery table. The table is partioned by date. When I need to run a backfilling script I iterate over a date range, get the data from some source (API in ...
jqurious's user avatar
  • 24.2k
1 vote
0 answers
31 views

I’m trying to use the Google Analytics Admin API (v1alpha) to link a GA4 property to BigQuery via the properties.bigQueryLinks.create method. Here is my code using : https://developers.google.com/...
Wendy Longuemare's user avatar
  • 11
17 votes
9 answers
30k views

I have a quite huge existing partitioned table in bigquery. I want to make the table clustered, at least for the new partition. From the documentation: https://cloud.google.com/bigquery/docs/creating-...
miken32's user avatar
  • 42.5k
0 votes
1 answer
642 views

How to implement Fuzzball JavaScript as a UDF inside Bigquery? Fuzzball has good amount of dependency libraries which is challenging to include as part of UDF inside Bigquery.
Mister Jojo's user avatar
  • 22.9k
0 votes
0 answers
79 views

I'm currently aware that I can export a BigQuery query result to Google Cloud Storage (GCS) by first creating a temporary table and then performing an extract table operation on that temp table. ...
AVA's user avatar
  • 183
0 votes
0 answers
62 views

I have got two BigQuery tables: bigquery:ssh-test-project-01.SSh_Dataset_03.SSh_BgQ_Src_01 and bigquery:ssh-test-project-01.SSh_Dataset_03.SSh_BgQ_Dst_BgQ_01. Using Data Transfer with SQL: "...
Ajeet Verma's user avatar
  • 4,523
-1 votes
1 answer
922 views

I'm trying to translate an SQL query into a polars code, and I'm stuck at one line of query, which basically join two tables with certain conditions, here is the sample SQL query that I have been ...
jqurious's user avatar
  • 24.2k
-1 votes
3 answers
98 views

I'm trying to run a simple SQL query in BigQuery like this: SELECT usertype, COUNT(*) FROM `project.dataset.table`; But I get an error: SELECT list expression references column usertype which is ...
Alparslan ŞEN's user avatar
  • 688
1 vote
2 answers
192 views

We're a long time users of the Google BigQuery INFORMATION_SCHEMA.TABLE_STORAGE view, never had problem using it in our automated data processes. But a couple of days ago even the simpliest SELECT ...
dreamca4er's user avatar
  • 444
1 vote
0 answers
100 views

Some of our BQ projects create external tables on top of parquet files like this: CREATE OR REPLACE EXTERNAL TABLE my_dataset my_table WITH PARTITION COLUMNS (ingestion_date DATE) OPTIONS ( format ...
Etienne Neveu's user avatar
  • 12.7k
10 votes
3 answers
17k views

How this error occurs "Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex" in google BigQuery
Bohemian's user avatar
  • 427k
1 vote
1 answer
9k views

I am trying to read data from BigQuery using pandas and pyspark. I am able to get the data but somehow getting below error while converting it into Spark DataFrame. py4j.protocol.Py4JJavaError: An ...
Community's user avatar
  • 1
0 votes
1 answer
46 views

I have the following code: SELECT h3s.h3id, h3s.geog, MIN(ST_DISTANCE(`carto-os`.carto.H3_CENTER(htsp.h3id), `carto-os`.carto.H3_CENTER(h3s.h3id))) OVER (PARTITION BY h3s.h3id) FROM ...
marc_s's user avatar
  • 760k
0 votes
0 answers
48 views

I need to define an external table from Google sheet in Dataform - only a few columns from a given sheet should be used. Some AI tools are saying that 'range' can be used in OPTIONS, but I guess it's ...
Sammy Merk's user avatar
  • 19
0 votes
0 answers
53 views

I’m using Cloud Composer (Airflow) and have two BigQuery operators like this: run_aggregation = BigQueryInsertJobOperator( task_id='aggregation_task', configuration={ "query":...
efesabanoglu's user avatar
  • 1
2 votes
1 answer
9k views

I'm saving the output of a model as a table in google big query from dataproc cluster using the below code: Rules.write \ .format("bigquery") \ .option("table","{}....
Community's user avatar
  • 1
0 votes
0 answers
65 views

I'm using dbt models to transform a QBO Balance Sheet uploaded as a CSV into Bigquery. Data comes from a fictional company inside QuickBooks test drives from which I export reports in CSV format. Here'...
almr27's user avatar
  • 53
1 vote
1 answer
62 views

I ran this standard query and it executes successfully. EXPORT DATA OPTIONS ( uri = 'https://pubsub.googleapis.com/projects/ggdevbigqueryproject/topics/GGTEST', format = 'JSON', ...
SANN3's user avatar
  • 10.2k
0 votes
1 answer
708 views

I am trying to run this query and having an issue with the "granted" field: SELECT ARRAY_TO_STRING(ARRAY(SELECT permission FROM t.protopayload_auditlog.authorizationInfo), ',') permissions,...
halfer's user avatar
  • 20.2k
1 vote
2 answers
29k views

I need to know how to implement this. I have written a SQL statement in big-query of this format: with a( select * from tab1), b as (select * from tab2), select <field.a>,<field.b> from a ...
Community's user avatar
  • 1
0 votes
0 answers
47 views

I have configured and put into production a BigQuery environment for one of our customers, and we have a question that we cannot answer, even after numerous analyses on our part. I configured the ...
Luca Magnotta's user avatar
  • 15
0 votes
1 answer
99 views

I have the below mentioned BigQuery SQL for a View. However, both the tables used in the query are huge in volume and hence I am facing terrible performance issues. If you'd glance at the query, I am ...
Baptiste's user avatar
  • 19
17 votes
7 answers
88k views

This is the query that I have been running in BigQuery that I want to run in my Python script. How would I change this/what do I have to add for it to run in Python? -- standardSQL SELECT Serial, ...
pjpscriv's user avatar
  • 1,546
2 votes
2 answers
9k views

I'm messing with avro file for the first time. I get a lot of csv files with the encoded data and just made the python treatment of the data that should go to BigQuery. In the end I got the schema ...
Community's user avatar
  • 1
1 vote
1 answer
1k views

I have a simple table (loaded from a view on top of files) with Countries and Zip codes in BigQuery. For some strange reason when both Country and Zip code are empty ... concat returns null instead of ...
RobMcZag's user avatar
  • 677
-1 votes
1 answer
88 views

I've installed the Firebase SDK in my iOS app and am using Google Analytics to measure data. The data is exported to BigQuery. The app incorporates consent mode for consent management. I was suddenly ...
marky's user avatar
  • 510
1 vote
1 answer
181 views

Description Hello, While trying to migrate a part of a code that works with Pandas, I fail to resolve the error below that the code gives me when using a Polars dataframe. In case of Pandas, I am able ...
Henry Shen's user avatar
  • 58

1
2 3 4 5
…
527