Index Now Using Google Api

Google Console

Go To Google Console

If its you first Time it Will be Similar to This (As of June 2024).
If Some Project is Slected then may be this.

On Search Bar Search new Project

Select Create a new Project (IAM and admin)

On New screen Fill Some Detail

you can give any name and Click Create
In an Moment On Top Right Of Screen will show some thing like below Image
Click On Select Project

Again Type Service Account On Search Section

Choose Service Account (IAM and admin)
Then on Following Screen Click Create Service Account
Fill The Details With Anything you like and Click Create and Continue
The in following Section From Drop Down Choose Currently used owner and click continue
Then After Click Done

Generate Crediential

On Following Screen it will show some thing Like this
Copy mailing adress which will be like xxx@xxx.iam.gserviceaccount.com and keep some place it will be need later
In action Click on 3 Vertical dot and then Manage Key
On Following Screen Click on add Key then create new key
A popup will appear select JSON and then Create

Download of JSON File

Search Bar New Project

Search Library on Search Bar

Select Library ( APIs and services)
Then On Next Screen Type Indexing and press enter or click search
On Following Screen Select Web Search Indexing API
Then On Following Screen Click on Enable

Google search-console

Go to Search Console

On Bottom Left Click Settings
Then Click on User and Permission
Click on Add User
Paste Email Address Which we copied Earlier in Google Cloud
For Referece Email Will Look Like this

In Permission Drop Down Select It As Owner if you Dont you get Error Some thing Like this

1
Error(403 - PERMISSION_DENIED): Permission denied. Failed to verify the URL ownership.

Code Section

Requirement

VS Code
PolyGlot Notebook Extension
.Net Sdk Installed
if Not Please Follow Installtion process.

Open VS Code

Create New Folder
Inside Folder Create three File
- apidetails.json
- data.txt
- indexing.ipynb

Open apidetails.json and Replace it Content With content of json file which was downloaded during Creating of Private Key will look like this

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
"type": "service_account",
"project_id": "xxxxx",
"private_key_id": "xxxxxx45dc588c3c433974e0eda",
"private_key": "-----BEGIN PRIVATE KEY-----\n\n-----END PRIVATE KEY-----\n",
"client_email": "xxx@xxx.iam.gserviceaccount.com",
"client_id": "106002683364xxxxx",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/xxxx.iam.gserviceaccount.com",
"universe_domain": "googleapis.com"
}

In data.txt File Fill all the URL in Follwing Fasion
Google Daily Limit Quata is 200.

Code in indexing.ipynb

Make Sure You Selected .Net Interactive
You don’t have to change Anything

Fill these data for Importing Library Fron Nuget Package

1
2
3
4
#r "nuget: Google.Apis.Auth"
#r "nuget: Google.Apis.Indexing.v3"
#r "nuget: Newtonsoft.Json"
#r "nuget: CsvHelper"

Click on Create next Cell

Importing relevant namespace which is used

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
using Google.Apis.Auth.OAuth2;
using Google.Apis.Indexing.v3;
using Google.Apis.Services;
using Newtonsoft.Json;
using CsvHelper;
using CsvHelper.Configuration;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;

Finall Code For Calling Google Api in next cell

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
string jsonKeyFile = "apidetails.json";
string[] scopes = { "https://www.googleapis.com/auth/indexing" };

GoogleCredential credential;
using (var stream = new FileStream(jsonKeyFile, FileMode.Open, FileAccess.Read))
{
    credential = GoogleCredential.FromStream(stream).CreateScoped(scopes);
}

HttpClient httpClient = new HttpClient(); 
var oauth = await credential.UnderlyingCredential.GetAccessTokenForRequestAsync();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", oauth);
var urls = File.ReadAllLines("data.txt");

foreach (var url in urls.Take(1))
{
    var result = await IndexUrl(url, httpClient);
    if (result.ContainsKey("error"))
    {
        Console.WriteLine($"Error({result["error"]["code"]} - {result["error"]["status"]}): {result["error"]["message"]}");
    }
    else
    {
        var metadata = result["urlNotificationMetadata"];
        var latestUpdate = metadata["latestUpdate"];
        Console.WriteLine($"urlNotificationMetadata.url: {metadata["url"]}");
        Console.WriteLine($"urlNotificationMetadata.latestUpdate.url: {latestUpdate["url"]}");
        Console.WriteLine($"urlNotificationMetadata.latestUpdate.type: {latestUpdate["type"]}");
        Console.WriteLine($"urlNotificationMetadata.latestUpdate.notifyTime: {latestUpdate["notifyTime"]}");
    }
} 

static async System.Threading.Tasks.Task<Dictionary<string, dynamic>> IndexUrl(string url, HttpClient httpClient)
{
    string endpoint = "https://indexing.googleapis.com/v3/urlNotifications:publish";
    var content = new
    {
        url = url.Trim(),
        type = "URL_UPDATED"
    };

    var jsonContent = JsonConvert.SerializeObject(content);
    var httpContent = new StringContent(jsonContent, System.Text.Encoding.UTF8, "application/json");

    var response = await httpClient.PostAsync(endpoint, httpContent);
    var responseString = await response.Content.ReadAsStringAsync();

    return JsonConvert.DeserializeObject<Dictionary<string, dynamic>>(responseString);
}

Click On Run All on to of VS Code. Wait for process to Finish

In Few Days Your Url Will be Indexed. For Quota limitation you can use Chuck URl by 200 for daily. Occial Doc For More or Short tutorial

Google Console#

Go To Google Console#

On Search Bar Search new Project#

On New screen Fill Some Detail#

Again Type Service Account On Search Section#

Generate Crediential#

Download of JSON File#

Search Library on Search Bar#

Google search-console#

Go to Search Console#

Code Section#

Requirement#

Open VS Code#

Code in indexing.ipynb#

Click On Run All on to of VS Code. Wait for process to Finish#